JOR: a content-based object router

Nader Mohameda,*, Xin Liub, Amy Davisb, Byrav Ramamurthyb

aElectrical and Computer Engineering, Stevens Institute of Technology, Burchard 212, Hoboken, NJ 07030, USAbComputer Science and Engineering, University of Nebraska, Lincoln, NE 68588-0115, USA

Received 5 August 2004; accepted 5 August 2004

Available online 11 September 2004

Abstract

Content-based routing has emerged as a new routing paradigm, allowing messages to be routed based on defined fields within the message.

Content-based routers generally employ XML, which has two main disadvantages. First, each message is translated into XML when sent, and

translated out of XML when received. Second, XML limits the objects sent to three types—data, documents, and messages. We introduce

here an extensible content-based object router that goes beyond messages to routing entire Java objects. The Java Object Router (JOR) is an

application-level router that allows Java objects to be routed according to their IP address, their label, their object type, or any of their

content. In addition, JOR provides mechanisms to deal with varying routing policies. JOR separates routing mechanisms from routing

policies, making it adaptable and easy to use in a variety of applications. To illustrate the advantages and performance of JOR, a prototype

was implemented to experimentally evaluate the content-based object routing mechanisms.

q 2004 Elsevier B.V. All rights reserved.

Keywords: Content-based router; Mobile object; Active networks

1. Introduction

The rise of object-oriented programming has resulted in

numerous distributed applications that require communi-

cation of objects. Furthermore, many distributed appli-

cations require flexible and extensible communication

infrastructures for content-based addressing and routing

[7]. For example, in grid computing, when a computational

problem is sent, it usually requires platform-specific

resources [16]. The computational object can contain a

field for the required platform type without specifying the

destination address, and thus should be routed based on the

content of that field. As a second example, in document

workflow systems, a document object may need multiple

approvals from different connected organizations or depart-

ments. These approvals are based on the document’s content

in which the object should be routed based on its content.

Mobile computing requires mobile agents to be commu-

nicated, and even simple client/server applications require

communication of request-reply objects.

0140-3664/$ - see front matter q 2004 Elsevier B.V. All rights reserved.

doi:10.1016/j.comcom.2004.08.008

* Corresponding author. Tel.: C1 201 216-5605.

E-mail address: nmohamed@stevens.edu (N. Mohamed).

In order to meet the demands of distributed applications,

several mobile object technologies transmit objects over

networks [12,21]. Java provides object serialization features

[13] in which object can be converted into a byte stream to

be sent on a network, and converted back into an object by

the receiver. Mobile object systems allow distributed

exchange of not only data but also code at run-time. One

example of utilizing objects on network is Java Object

Passing Interface (JOPI) [17], which provides an MPI-like

interface [22] that can be used to exchange objects among

distributed processes. Another example is distributed

system middleware [2,3], which allows different types of

objects to be used as controls, computations, requests or

replies among the distributed system components.

We introduce a flexible and extensible content-based

object router architecture that meets the demands of such

distributed applications as grid computing and workflow

systems while combining the advantages of several mobile

object systems. This architecture allows entire objects to be

routed transparently between computers, regardless of type,

content or platform. We provide a Java implementation, the

Java Object Router (JOR), as a proof of concept for

content-based routing of objects, separation of mechanism

Computer Communications 28 (2005) 654–663

www.elsevier.com/locate/comcom

Fig. 1. Two clusters are connected through the head nodes.

N. Mohamed et al. / Computer Communications 28 (2005) 654–663 655

and policy, and routing transparency at the application level.

We employ techniques developed in active networks [1,5,6,

15,18,23] to provide flexible network infrastructures for

new content-based services and applications.

JOR is an application-level router designed to allow any

type of Java object to be transmitted and routed from one

Java application to another over a TCP/IP network. It allows

users to route objects based on header fields (like source IP

addresses, or a label),

or on content (like type of object being sent or some

properties of the object). Inside JOR resides weighted fair

queuing (WFQ) scheduling, which allows the user to

implement different priority requirements. Because JOR is

modular, separating the routing mechanism from the routing

policies, it allows the user to implement a variety of policies

without changing JOR itself through the use of Java code

configuration. Therefore JOR gives the user an opportunity

to easily enforce routing policies for different applications.

This paper first discusses content-based object routing

concepts in Section 2. Section 3 then describes the

architecture of JOR. Some applications of JOR are

described in Section 4, followed by an evaluation of the

performance of JOR in Section 5. Section 6 discusses the

related work and Section 7 presents concluding remarks and

discusses future work.

2. Content based object routing

Content-based routing has gained strong ground by

allowing documents to be communicated between compu-

ters—even with disparate software. This capability has

allowed content-based routing to become not merely an

interesting research topic, but a useful business paradigm.

However, to date, most content-based routing approaches

have been concerned with the routing of messages. XML

has arisen as a popular method for routing messages. The

messages are coded into XML and routed via an XML

content-based router [9–11]. In such systems, both the

router and the end user, or server program, understand the

XML tags. By coding an XML router. The processes of

sending and receiving can be made slightly more transparent

to the end user. In particular, with the proper set of tags, an

XML router can send a document to any number of different

places depending on various properties in the message.

Advanced programming languages such as Java provide

machine independent execution, which make them useful

for applications running on heterogeneous systems.

Examples of these applications are Grid, Web, and mobile

code applications. Java provides object serialization fea-

tures [13] in which object can be converted into a byte

stream. When a class is serializable, any object instantiated

from that class can be stored or transmitted on the network

to any remote machine. An object byte stream contains all

the required information to reconstruct the object. The

receiving process on the receiving machine can change the

value of any attribute or invoke any method in the received

object. Object serialization allows distributed applications

to exchange objects at run-time.

JOR not only allows the exchange of objects within a

distributed system but also does not require the use of a

specific destination address. We already noted that this is

useful in grid computing and document workflow systems.

For a third example, consider a network of two switched

clusters connected only by the head nodes (HN) [3], as

shown in Fig. 1. If processes in this network each have their

own address and need to exchange objects, then each object

exchanged must be through the head nodes. Many parallel

programming environments use the process ID to identify

both the sending and receiving processes. An architecture

like JOR permits these objects to be easily routed to the

correct destination, despite an unknown destination address.

Other smaller examples are the routing among peer-to-peer

systems and overlay networks in which some peers or nodes

are connected through some other peers or nodes.

We extend the concept of content-based routing to

include any Java object seamlessly, and the ability to route

based on any method or field in that Java object.

Additionally, we provide a generic routing tool that contains

all mechanisms necessary to route any Java object based on

any property of that object. In so doing, we present a

technique for separating the mechanisms that perform the

routing of an object from the routing policies for many

applications. Because we provide the underlying routing

mechanism, declaration of a routing policy becomes a

simple and straightforward exercise. Therefore, we provide

flexibility in JOR to support a number of different

applications with minimal changes to the actual code.

3. JOR architecture

The design goal of JOR is to provide a flexible and

extensible communication platform for distributed appli-

cations by separating the mechanism of object routing from

policies of that routing. In order to meet these goals, JOR

supplies internally the mechanism to route objects and

externally an interface for the users to customize specific

routing behavior. The general functionality of JOR is

merely to accept objects and send them out to the

appropriate destination, while the user defined routing

rules choose the destinations. Unlike the network routing

Fig. 2. JOR architecture.

N. Mohamed et al. / Computer Communications 28 (2005) 654–663656

rules, application-level routing rules can be based on higher-

level criteria such as workload balancing, object contents,

fault-tolerance, and workflow.

Generally, JOR consists of four components: the object

receiver, the object manager, the object queues, and the

object dispatcher, as shown in Fig. 2. The receiver receives

objects from sending applications and passes them to the

object manager. An object manager is simply a repository of

rules, defined by the user, for routing and handling objects

from an application. These rules define how the incoming

object should be handled based on any criteria, including

invoking methods carried inside the object, changing the

object itself (similar to IP masquerade) or the contents of

fields, or directly passing the object to the queue. The object

queues schedule incoming objects based on user-custo-

mized rules, such as weighted fair queuing (WFQ) [8] or

first in first out (FIFO) queue. The dispatcher removes the

object at the queue head and checks with the object manager

if there is any action should be done before sending the

object to its destination.

3.1. JOR receiver and dispatcher

Like any XML router, JOR can receive messages, and

send them to the appropriate destination(s). Unlike any

XML router, JOR permits any Java object, not just

documents, to be routed. In order to send an object to

JOR, the sending application uses Java serialization to

transform the object into a bit stream, and sends it to the

JOR router. It is only required that the sending application

know the address of the JOR router. The router can be

configured to detect the proper destination based on the

object itself.

Both the receiver and the dispatcher have a thread

pool. The numbers of receiver and dispatcher threads are

configurable. When an object arrives, JOR picks a

receiving thread in the receiving thread pool to serve

the object. Each object, when entering the JOR router, is

wrapped in an objectPacket (alternatively, the

sending application may pre-wrap an object in the

objectPacket for JOR). This objectPacket is

the native object type for JOR and includes fields for the

source and destination addresses, labels, priorities, and

information on the enclosed object (its type and length,

for example).

After wrapping the incoming objects, the receiver

forwards them to the object manager for content-based

processing, as described in Section 3.2. The object may

eventually be forwarded to the object queue for dispatch.

When this is the case, the dispatcher removes the object at the

queue head at send-time, unwraps the object back to its

original type, and checks with the object manager if there is

any action should be done before it sends the objects to the

destination.

3.2. The object manager and user configurations

Inside JOR, the object manager receives two types of

Java objects from the receiver—a configuration object and a

communication object. A configuration object installs

routing policies to serve a distributed application. One

instance of JOR can support multiple configuration objects

and therefore serve multiple distributed applications. Each

configuration object carries Java bytecodes for a set of

classes (the classes of communication objects supported by

this configuration object). Additionally, the configuration

object carries a handling object and its class bytecodes. The

handling object is a place, where the user can define

different handlers (methods) for different object types or

handlers for different source IP.

Communication objects are, of course, those used in

communication among the distributed applications, and are

generally just in transport through JOR (rather than

modifying the behavior of JOR). JOR must receive at

least one configuration object before it is capable of sending

and receiving communication objects. However, a com-

munication object may change the status of the responsible

handling object, i.e. changing a field in the handling object.

This approach of using two types of objects is similar to the

discrete approach used in active networks [6] in which the

configuration object is equivalent to the management packet

and the communication packet is equivalent to the datagram

packet.

When the object manager receives a configuration object,

it updates a list of total supported communication objects

based on that configuration, and stores both the bytecodes of

the supported classes as well as the handlers for those

classes. The bytecodes of supported classes is needed to

enable JOR to de-serialize and handle the supported

communication objects. The JOR class loader extends the

standard Java virtual machine class loader [13] by allowing

the user defined classes for communication objects and

handling policy to be loaded from the configuration object.

When the object manager receives a objectPacket, it

passes it to the proper handling class. The object manager

runs existing handlers based on the source IP address, the

label, and the class type of the communication object (inside

the objectPacket), in that order. If any handler is

defined for these, they are invoked. In these handlers, which

are also Java code, users can define and perform actions

Table 1

N. Mohamed et al. / Computer Communications 28 (2005) 654–663 657

such as:

Configurable features of JOR

Event Config. Level Customizable actions

–

Initialization JOR Level Number of listening and

Defining next hop or destination address for the incoming

communication object

sending threads

–

Listening ports

Invoking a method (or methods) in the incoming

communication object

Number of queues – Changing the incoming communication object fields. Max number of delivery –

Object entrance Application level

attempts

Default inMethod

Changing the responsible handling object state (i.e.

changing the contents of field(s) in the handling object).

Default priority

–

Default destination IP

address

Passing the incoming communication object or any new

generated communication objects to the objects queue

for forwarding.

Object specific type – inMethod

Source IP specific

inMethod

Label specific inMethod

Object exit Application Level Label specific outMethod

Destination IP specific

outMethod

Object specific type

outMethod

Default outgoing priority

Default outMethod

Exception handlers JOR Level

fullBuffer DefaultZtaildrop

noDestinationIP DefaultZdrop

deliveryError DefaultZretry maxAttempts

times, then drop

default Configure to handle other

exceptions

Defining in which queue of the WFQ the incoming

communication object or any new generated objects need

to be stored.

Various handlers are also checked for and executed when

an object is dispatched. First, the default outMethod is

invoked, if one is defined, followed by methods based on the

destination IP address, the label and the object type. Before

exiting towards its next hop or destination, the object is

returned to the type that it has arrived in. If it arrived as a

String, it leaves as a String. If it arrived as an object-Packet, it leaves as an objectPacket. However, the

user can override this default behavior.

Because JOR separates the routing mechanism from

routing policies, the routing rules are completely customiz-

able and configurable by the user. In the handling class, the

user can define a set of methods to deal with the incoming

communication objects as well as maintain any state needed

for the routing process by defining and using an attribute in

the handling object. The user can define specific methods to

execute at the arrival or departure of any communication

object. These rules are stored in a handling class in which

users can define different types of handlers using Java code.

For example, the user can define a method named

‘in_Transaction’ in a handling object. This method will be

used as a handler for any object of type ‘Transaction’

received by JOR. In addition, the user can define a method

‘in_192.168.1.10’ to be a handler for any object received

from IP address 192.168.1.10. In these handlers the user can

work on the object or check its contents to define the

destination address. Similarly handlers for object exit can be

defined. This component can contain anything from the

simplest logic (defining defaults), to complex functions and

method invocation. Table 1 shows the possible areas to

customize. Because handler classes are implemented in

Java, JOR can be configured in endless ways for many

networking applications.

Each of the error handlers and default functions come

pre-configured in JOR, but any user may define their own,

overriding default behaviors. Additionally, each running

instance of JOR may be associated with specific configur-

ation objects, allowing different JOR programs, performing

different tasks, to run simultaneously. Therefore, JOR

achieves modular separation of the routing mechanisms

and the policies implemented. JOR provides the routing

mechanisms, and gives the user an opportunity to enforce

routing policies for different applications by defining

different configuration objects and sets of handlers.

The object manager uses the reflection API [13] to invoke

the handler methods defined by the user in the handling

class. With the reflection API, it is possible to determine the

class of an object and get information about a class’s

modifiers, fields, methods, constructors, and superclasses. It

is also possible to find out what constants and method

declarations belong to an interface, create an instance of a

class whose name is not known until runtime and get and set

the value of an object’s field, even if the field name is

unknown to your program until runtime. In addition, one can

invoke a method on an object, even if the method is not

known until runtime, and create a new array, whose size and

component type are not known until runtime, and then

modify the array’s components. JOR utilizes these features

to execute handlers in the object manager.

3.3. Object queues

The received objects are stored in the object queues

before dispatching. The queue component provides the

‘enqueue’ and ‘dequeue’ operations to the receiver and the

dispatcher. There are two types of object queues: FIFO

N. Mohamed et al. / Computer Communications 28 (2005) 654–663658

queue and WFQ [8]. These queues can be customized by the

user to provide different quality of service (QoS) policies.

In this work, we implement the WFQ algorithm in the

default queue component. Based on the JOR configuration

file, the user can create a set of independent queues and

assign a weight to each queue. Each queue is associated

with a virtual finish time. When an object pi is put into

an empty queue qj, its virtual finish time is calculated

as follows:

ftj Z enqueue timeðpiÞCsize ofðpiÞ

weightj

When an object is delivered, the virtual finish time is

updated as follows:

ftj Z ftj Csize ofðpiC1Þ

weightj

The WFQ scheduler always selects the queue with the

earliest expected finish time.

When an incoming object arrives at the object manager,

the proper method in the corresponding handler class

defined by the user will be invoked. In the invoked method,

the user can dictate which queue the object should be stored

before calling the ‘enqueue’ method. Therefore, the user can

choose queue destination based on the type or content of the

object. The WFQ will prioritize the enqueued objects based

on the defined weight of its queue.

Two examples illustrate this. First, we can classify the

objects by the source–destination combination by dividing

traffic into several groups based on source and destination

Ips: group A going from subnet 129.93.33.0/24 to subnet

129.93.68.0/24, and a group B going from subnet

129.93.65.0/24 to subnet 129.93.68.0/24. Here, the traffic

of group A will not affect the traffic of group B. We can

forward the communication objects to specific queues based

on their group. Although in this example we use the source

IP and destination IP to define to which queue the object

should be forwarded, JOR also allows other fields in the

object to identify which queue to be used. As a second

example, consider an e-commerce application in which

purchases of high amounts should be quickly delivered. The

user can implement java code to check the transaction

amount in an incoming transaction, and assign each

transaction a queue based on that amount. While this is a

deviation from standard WFQ, it allows JOR to assign

different priorities to different objects based on their type,

source, destination, contents, or combinations of these

properties.

Fig. 3. Using JOR for load balancing and fault tolerance.

4. Applications

The strength of JOR lies in the many applications that

can use it. Because JOR is modular, it is capable of

providing a variety of distinct services to the user based on

the logic stored in the configuration object. Therefore, any

user of this product can implement their application policies

with limited programming. The JOR router can either reside

on one of the machines executing the distributed application

or on a dedicated machine. Distributed applications can use

one instance of JOR or a network of JORs for their object

routing needs. JOR can be used as a building block to

implement many other services for distributed systems. In

addition to load balancing, fault tolerance, and workflow

management, it can be used to route mobile objects based on

its contents. JOR can be configured to route computational

objects in cluster and grid computing to suitable compu-

tational machine or sites. In addition, content-based

addressing and priority assignment, weighted fair queues,

and extensibility features of JOR provide flexible mechan-

isms to implement different types of differentiated services

for object routing.

In Section 4.1, we discuss a simple example application.

In Section 4.2 we discuss an extended application and

Section 4.3 discusses an application in cluster and Grid

computing. We show the performance of JOR in Section 5.

4.1. A simple application: fault-tolerant server pool

A simple configuration of JOR to provide load

balancing or fault tolerance is discussed below. For

example, JOR can be configured to maintain a server

pool. Each time a user object arrives, JOR gets a server

from the server scheduler and forwards the object to that

server. When a new server starts, it sends JOR a

‘newserver’ object packet. Then JOR invokes a registration

handler to add a server entry into the server pool. If the

server is unable to receive the packet, it removes the

corresponding server entry. This application is shown

conceptually in Fig. 3. In the configuration file, the user

needs to use the routing mechanism provided by JOR to

implement the routing policy as well as implement logic

for both registration handler and server scheduler. Since all

routing mechanisms are provided by JOR, the user can

concentrate on implementing the logic and policy of

routing objects for the network applications.

N. Mohamed et al. / Computer Communications 28 (2005) 654–663 659

4.2. Graduate student application process: a workflow

application

To demonstrate the flexibility, scalability, and ease-of-use

of the JOR system, an example is given here regarding

graduate student admission process. Students can apply

on-line for graduate studies through the web. The web

application form contains a servlet responsible for creating a

graduate admission Java object that contains all information

provided by the student. This object needs to be transferred

among Java applications belonging to different departments

within the university during the application process. The

graduate admission application class is as follows:

A graduate application needs to be checked and approved

by many departments (such as graduate studies office,

international affairs, and academic departments). Addition-

ally, some information may be requested of the student

during the admission process. The application needs to pass

through approval steps starting from graduate office initial

checking, academic department acceptance, and finally

international affairs office for financial checking and

approval for international student applications (see Fig. 4).

There are two ways to implement the communication and

routing processes for the application object among different

systems:

Fig. 5. Routing policy for a graduate admission application object.

1.

F

Let the distributed applications communicate directly

and implement the routing policy within all applications.

ig. 4. JOR routes Java objects among distributed Java applications.

2.

Use a content-based routing such as JOR to implement

routing policies (see Fig. 5).

The advantage of the first solution is avoiding a single-

point of failure. However, there are many disadvantages

such as:

1.

It is difficult to add QoS policies.

2.

Routing policies are scattered among different appli-

cations; therefore, managing and maintaining these

policies is a complex task.

3.

Changing the routing policies may involve changing one

or more distributed applications, which may require

access to the source code for the applications.

N. Mohamed et al. / Computer Communications 28 (2005) 654–663660

The advantages of using the second solution are:

1.

There is a well-defined separation between the routing

policies and the routing mechanism. As a result,

changing the routing policies is an easy task. All changes

can be done in JOR setup only.

2.

It is easy to add QoS to the routing policies.

3.

There is no need for having or changing the source codes

of the distributed applications to change the routing

policies.

4.

It is easy to add new functions to the distributed

application because JOR can be used as a class format

converter between distributed applications.

Using JOR, the routing policies of the graduate

admission application are easily defined. The handling

class in the configuration object implements the routing

policies of the object among distributed applications. The

user can easily define destination addresses based on the

object contents. In addition, other setup can be defined, such

as object processing priority and error handlers.

The handling class for the Graduate Admission is as

follows:

4.3. Object routing in grid computing

This section describes applications of JOR in multiple

cluster and Grid environments. As discussed in Section 2,

nodes from different clusters or sites may not see each other

directly. However, the nodes from different clusters or sites

can communicate through the cluster head nodes. In order to

run a parallel job that spans multiple cluster or unconnected

machines, a routing mechanism is needed among these

nodes. The JOR can be installed in the head nodes of

multiple clusters to enable the communications among

nodes from different clusters.

One example of using JOR in a parallel environment that

utilizes multiple heterogeneous machines is the Java object

passing interface (JOPI) [17] that runs on top of distributed

run-time environments [2,3]. Here, a parallel job with

multiple processes can be scheduled to run on machines

from different clusters. The processes of a parallel job

communicate by using Java objects, and are identified by

process id (PID). The distribution of processes is dynami-

cally assigned. In order to facilitate process communicate

among machines, JOR is installed in the head nodes of each

cluster, and a configuration object is sent to each instance of

JOR. The configuration object contains the code of the

routing process for JOPI applications. The parallel environ-

ment scheduler sends mapping table object to all the

instances of JOR involved in the processing before starting

the parallel job. This mapping object contains a mapping

table to map PID to IP/Port address. Based on this table,

JOR routes JOPI communication objects among the

environments. After completing the parallel job, the

scheduler sends another object to remove the routing table

from JOR.

5. Performance

Four types of experiments were conducted to evaluate

different aspects of JOR. The first experiment in Section 5.1

is to evaluate routing time and overhead of different Java

object types. In Section 5.2, the throughputs of JOR using

different Queue types are evaluated. In Section 5.3, the

performances of JOR and RMI on the Internet are

compared. Finally, Section 5.4 evaluates an application

that uses JOR for object routing.

5.1. Object routing time and overhead

Three micro benchmark programs were written to

evaluate the performance of JOR. The experiments were

conducted on two dual 1.2 GHz AMD-processor machines,

with 256 KB cache per processor and 1 GB RAM per node.

The nodes were connected via a 100 Mbps Ethernet.

For these experiments, the standard JVM version SDK 1.4

was used. To measure the JOR performance, the client and

server ran on the same machine while JOR ran on another

machine. The experiments were designed to measure JOR

routing delay with different types and sizes of objects. Three

different types of object have been used, as shown in

Table 2. In general, this overhead is acceptable given the

benefit of using JOR to facilitate routing among distributed

applications.

Fig. 7. Throughput of JOR with WFQ queuing.

Table 2

Routing times of different object types

Object type Size

(bytes)

Total transport

time (ms)

Routing over-

head time (ms)

Double 84 1 !0.5

Graduate application 739 2 !0.5

Hashtable object

(8000 integers)

135,041 123 1

The transport time includes object serialization, communication, routing,

and deserialization.

N. Mohamed et al. / Computer Communications 28 (2005) 654–663 661

5.2. Performance of JOR queues

The behavior of the WFQ component is compared with

a basic FIFO queue in our experiments. We created three

clients in both cases. Client1 sends objects from 127.0.0.1

to O127.0.0.1; Client2: 129.93.33.1 to O129.93.33.1;

Client3: 129.93.33.1 to O129.93.46.177. Client2 and

Client3 were sending objects at the same speed while

Client1 was twice as fast. We consider that the object

contains two fields for source IP and destination IP. In the

FIFO implementation, all objects were put in the same

queue. In the WFQ implementation, however, an indepen-

dent queue was created for each connection and we

assigned a weight to each connection, Client1: weight 10,

Client2: weight 50, Client3: weight 40. Figs. 6 and 7 show

the throughput under both the FIFO queuing algorithm and

the WFQ queuing algorithm. We can see that the objects

were delivered at the expected speed. In the WFQ case,

Client1 and Client3 performed better when Client2

terminated, and Client1 finally utilized the whole band-

width when Client 3 terminated.

5.3. Comparing JOR and RMI on the internet

RMI [19] provides the necessary interface to access remote

methods in a similar manner to remote procedure calls (RPC).

RMI was introduced in Java to hide the communication

details and provide the developer with a remote method

Fig. 6. Throughput of JOR with FIFO queuing.

invocation interface. RMI allows applications to locate and

communicate with remote objects, and to load class bytecode

for objects that are passed around. Some available RMI

systems can be adapted to handle Java objects, but they do not

provide seamless integration with the Java system because

they have to deal with multiple languages. An example is

CORBA, which is designed to deal with different languages;

therefore, it has a language-neutral object model. In contrast,

RMI is built specifically for Java and takes advantage of the

Java object model. Serializable objects can be transferred as

parameters or as return values between distributed Java

applications. Since RMI is the most recommended com-

munication mechanism among distributed Java applications,

we compare the performance of JOR and RMI on the Internet.

RMI was built to support distributed applications, but it is

certainly not very suitable for content-based routing appli-

cations. This is because RMI’s client programs need to know,

where the remote RMI object is located in order to invoke a

method in the object.

We try to pass two types of objects between a client

machine on the Internet (off-campus) to another machine

(router) in the University of Nebraska–Lincoln (UNL)

campus. Then the objects were routed to a server machine

also on campus. We used two routing machines, one using a

routing program built from scratch on RMI and another

using JOR with user routing policy and FIFO queue. We

measured round trip times (RTT) for the communicated

objects. The transfer time is calculated as half RTT. Table 3

shows the results. Although JOR deploys a FIFO queue,

Table 3

JOR and RMI routing performance

Object type Average end-to-end

transfer time (JOR)

(ms)

Average end-to-end

transfer time RMI

(ms)

Double (Java class) 33.146 33.38

Graduate application

(user defined class)

36.770 38.589

Table 5

The performance of TSP on different configurations

No. Configuration Object routing

involvement

Execution

time (s)

N. Mohamed et al. / Computer Communications 28 (2005) 654–663662

it can provide better routing time. This is due to the fact that

JOR uses standard socket for communication that provides

better performance than RMI while JOR queue can add

reliability for the communication.

1 Sequential TSP on a single

processor of the cluster

No 35609.73

2 Sequential TSP on a single

processor of CSNT

No 37839.89

3 Parallel TSP on two nodes of

the cluster (4 processors) and

2 on CSNT

Yes 7782.309

4 Parallel TSP on three nodes

of the cluster (6 processors)

and 2 on CSNT

Yes 5633.393

5.4. Performance of a JOR application

As described in Section 4.3, JOR was used to route JOPI

objects among unconnected multiple clusters and hetero-

geneous systems. An experiment was conducted to measure

parallel traveling salesman problem (TSP) using JOPI. The

parallel TSP algorithm was based on branch-and-bound [14].

The problem was implemented by using four types of user-

defined objects. An object was used to handle the search

mechanism, given the proper range in the problem and a

bounding value (minimum current tour value). Another object

type was used for reporting the result tour by different process

to master process. The minimum tour value and the search

range were also represented as objects. Broadcast was used to

distribute the original problem object to the processes. Each

process performed the search within its range, reported the

local best tour to the master process and requested a new range

for next search. In addition, the processes broadcast the local

minimum tour value found. This allowed other processes to

update the minimum value to speedup their search. The object

routing used among processes ran on indirectly connected

machines. A 22-cities TSP problem was run in different

heterogeneous machines. The machines are described in the

Table 4. The cluster’s nodes are connected through a Fast

Ethernet switch while the cluster head node and CSNT are

connected through another fast Ethernet network. A JOR was

activated on the cluster head node to route JOPI application

objects among the cluster nodes and CSNT. The execution

times of running the TSP problem under different configu-

rations with and without object routing are shown in Table 5.

Therefore, as demonstrated by this application, JOR enables

JOPI to work in multiple clusters.

The network interface that connects the head node in the

cluster with machines outside the cluster represents a

bottleneck for the network traffic going to or coming from

the cluster. To test the performance of parallel TSP with the

existence of other network traffic in and out of the cluster,

parallel TSP was run using the fourth configuration in Table 5.

Two sets of experiments were conducted: one with FIFO

queue and one priority queue. Both sets were conducted

during generating other network traffic from and to the cluster.

The priority queue in JOR was configured to give high priority

Table 4

List of machines used in experiments

Name Platform description

CSNT 3 CPU, Intel !86 700 MHz, 1.5 GB RAM OS:

Windows 2000 adv. Server

Cluster 24 nodes, dual 1.2 MHz AthlonMP, 256 KB cache,

1 GB RAM per node. OS: Linux

for TSP objects and low priority for other communication

objects. The average execution time of running the same

22-cities TSP objects using FIFO queue in JOR was

6459.274 s. The execution time of running the problem

using JOR with priority queue was reduced to 5879.847 s.

6. Related work

XML routing can be found in TIGRA [10] and YFilter [9].

TIGRA is an architecture style for enterprise application

integration. YFilter provides efficient and scalable filtering of

XML documents. XML routing can be also used to filter

XML for Web services [11]. Microsoft’s BizTalk [4] provids

a pre-defined and domain specific set of XML tags for various

sets of documents. Both the client and the server translate

their documents into this XML for sending, and back from

the XML for receiving. Sundsted [20] demonstrated how to

modify the Java messaging service (JMS) to support routing

of Java objects by properties other than the address-that is, by

content. It uses an XML message format to encode messages

so that a router written in Java can parse those same messages

to determine the proper destination of the message. Inside the

router are coded rules that specify, where particular

information must go. A router rule is composed of XML

tags to look for, and a destination specifying, where the

document goes depending on the tags found. Because both

the object sent and the router are written in Java, the

conversion to and parsing from XML is an added step.

While the purpose of the JMS [20] is to route messages,

we note that if the conversion to XML were avoided, entire

objects could be received, and sent on by the router. Thus, the

router is extended from routing messages to routing objects,

and improved by avoiding the conversion to and from XML.

Finally, using very similar code rules, we have the same (and

greater) ability to route based on the content of an object. In

particular, we have the ability to execute code in the object at

any router, creating a mobile code environment. In addition,

JOR provides an active platform, where new functions and

service can be added easily for content-based routing.

Magician [15] uses the technique of serializable Java objects

N. Mohamed et al. / Computer Communications 28 (2005) 654–663 663

for presenting active packets in active networks. However,

magician has a general architecture for active networks and is

not designed to provide a flexible platform for content-based

object routing for distributed applications.

7. Conclusion and future work

JOR is an application-level router that allows Java objects

to be routed according to their IP address, their object type, or

any of their content. JOR utilizes the techniques developed

for active networks to provide flexible and extensible

communication routing for content-based distributed appli-

cation. This flexibility is achieved through object trans-

parency, configurability and modularity. As long as the

distributed applications at either end of the network under-

stand the types of objects that are sent and received, it is not

required that JOR have any knowledge about the objects.

However, JOR can be configured to have knowledge about

objects, and their contents. Because the JOR configuration is

implemented in Java, it can be configured in endless ways for

almost any type of networking application. Therefore, JOR

achieves modular separation of the routing mechanisms and

the policies implemented. JOR provides the routing mech-

anisms, and gives the user an opportunity to enforce routing

policies for different applications.

As part of our future work, we will investigate the

security mechanisms for content-based object routing. In

addition, we will investigate the mechanisms to support

object delivery using the publish/subscribe model, which is

a very useful approach for delivering a high number of

messages to a large number of receivers.

Acknowledgements

A preliminary version of this paper was presented at the

14th International Conference on Parallel and Distributed

Computing and Systems, Cambridge, Massachusetts, 2002.

The fourth author was supported in part by a US National

Science Foundation grant (EPS-0091900) and the UNL

Program of Excellence PRISM Priority Initiative. We would

like to thank Jameela Al-Jaroodi for her valuable comments

and reviews of this paper. In addition, would like to thank

the research computing facility (RCF) at UNL for allowing

us to use their facilities for some of our experiments.

References

[1] D. Alexander, W. Arbaugh, M. Hicks, P. Kakkar, A. Keromytis,

J. Moore, C. Gunter, S. Nettles, J. Smith, The switchware active

network architecture, IEEE Network 12 (3) (1998) 27–36.

[2] J. Al-Jaroodi, N. Mohamed, H. Jiang, D. Swanson, An Agent-Based

Infrastructure for Parallel Java on Heterogeneous Clusters, in

Proceedings of the fourth IEEE International Conference on Cluster

Computing, Chicago, Illinois, IEEE, September 2002, pp19–27.

[3] J. Al-Jaroodi, N. Mohamed, H. Jiang, D. Swanson, Middleware

Infrastructure for Parallel and Distributed Programming Models on

Heterogeneous Systems, IEEE Transactions on Parallel and Dis-

tributed Systems—Special Issue on Middleware, Guest Editors:

Rachid Guerraoui and Willy Zwaenepoel, 2003, 14(11): 1100–1111.

[4] BizTalk.org Page, http://www.microsoft.com/biztalk/, 2003.

[5] S.F. Bush, Active virtual network management prediction: complexity

as a framework for prediction, optimization, and assurance, DARPA

Active Networks Conference and Exposition (DANCE’02), IEEE

Computer Society Press, May 2002. pp. 534–553.

[6] S.F. Bush, A.B. Kulkarni, Active networks and active network

management: a proactive management framework, Kluwer Aca-

demic/ Plenum Publishers, New York, 2001.

[7] A. Carzaniga, D.S. Rosenblum, A.L. Wolf, Content-Based Addres-

sing and Routing: A General Model and its Application, Technical

Report CU-CS-902-00, Department of Computer Science, University

of Colorado, Boulder, Colorado, January 2000.

[8] A. Demers, S. Keshav, S. Shenker, Analysis and simulation of a fair

queuing algorithm, Journal of Internetworking Research and Experi-

ences October 1990; 3–12.

[9] Y. Diao, P. Fischer, M.J. Franklin, R. To, YFilter: efficient and

scalable filtering of XML documents, 18th International Conference

on Data Engineering (ICDE’02), 2002 pp. 341–342.

[10] W. Emmerich, E. Ellmer, H. Fieglein, TIGRA—an architectural style

for enterprise application integration, 23rd International Conference

on Software Engineering (ICSE’01), 2001 pp. 567–576.

[11] P. Felber, C.-Y. Chan, M. Garofalakis, R. Rastogi, Scalable filtering of

XML data for web services, IEEE Internet Computing January 2003;

49–57.

[12] A. Fuggetta, G. Picco, G. Vigna, Understanding code mobility, IEEE

Transactions in Software Engineering 24 (5) (May 1998) 342–361.

[13] Java API Specification Website: http://java.sun.com/reference/api/

index.html, Sun Microsystems, 2004.

[14] R. Karp, Y. Zhang, Randomized parallel algorithms for backtrack

search and branch-and-bound computation, Journal of the ACM vol.

40 (3) (July 1993).

[15] A.B. Kulkarni, G.J. Minden, R. Hill, Y. Wijata, S. Sheth, H. Pindi,

F. Wahhab, A. Gopinath, A. Nagarajan, Implementation of a

Prototype Active Network, OPENARCH, San Francisco, 1998.

[16] B. Liljeqvist, L. Bengtsson, Grid Computing Distribution Using

Network Processor, Proceedings of the 14th International Conference

of Parallel and Distributed Computing and Systems, Cambridge,

USA, November 2002, pp. 12–17.

[17] N. Mohamed, J. Al-Jaroodi, H. Jiang, D. Swanson, JOPI: a Java

object-passing interface, in: Proceedings of the Joint ACM Java

Grande-ISCOPE Conference (JGI2002), Seattle, Washington, ACM,

November 2002. pp. 37–45.

[18] N. Mohamed, J. Al-Jaroodi, Active Networks for Enhancing Grid

Operations, in Proceedings of the International Workshop on Grid

Computing and e-Science in conjunction with the 17th Annual ACM

International Conference on Supercomputing (ICS’03), San Fran-

cisco, California, June 2003.

[19] Sun Microsystems. RMI Specification, http://java.sun.com/products/

jdk/rmi/.

[20] T. Sundsted, Using the JMS API and XML in content-based routing:

how to make sure information gets to the right people. http://www-

106.ibm.com/developerworks/library/j-jms/?n-j-1120.

[21] J. Vitek, C. Tschudin (Eds.), Mobile Object Systems: Towards the

Programmable Internet, Lecture Notes in Computer Science No.

1222, Springer, Berlin, April 1997, pp. 111–130.

[22] D. Walker, J. Dongarra, MPI: a standard message passing interface,

Supercomputer 12 (1) (January 1996) 56–68.

[23] D. Wetherall, J. Guttage, ANTS: a toolkit for building and

dynamically deploying network protocols, in: Proceedings of OPEN-

ARCH, San Francisco, April 1998.