Security Analysis of Firewall Rule Sets in Computer Networks

Bilal Khan Center of Excellence in Information Assurance (CoEIA), King Saud University, Saudi Arabia.

Email: bilalkhan@ksu.edu.sa Maqsood Mahmud

Center of Excellence in Information Assurance (CoEIA), King Saud University, Saudi Arabia.

Department of Information System, CCIS, King Saud University, Saudi Arabia.

Department of Information System, University Technology Malaysia, Malaysia. Email: maqsood.m@ksu.edu

Muhammad Khurram Khan Center of Excellence in Information Assurance (CoEIA),

King Saud University, Saudi Arabia. Email: mkhurram@ksu.edu.sa

Khaled S. Alghathbar

Center of Excellence in Information Assurance (CoEIA), King Saud University, Saudi Arabia.

Department of Information System, CCIS, King Saud University, Saudi Arabia.

Email: kalghathbar@ksu.edu.sa

Abstract - Firewalls are the screening gates for the internet/intranet traffic in computer networks. However, deploying a firewall is simply not enough since it needs to be configured by the system administrator according to the needs of the organization. There are many reasons due to which it is hard for the administrator to configure the firewall properly. Specifying firewall rule set is complicated and error prone. Once the firewall rules are defined, then firewall should be tested, whether it actually implements firewall policy. In this paper, one of the approaches of the firewall rule set analysis, i.e., the problems with the structure of the firewall rule set is being addressed. The structure of a sample firewall rule set is analyzed to detect and resolve conflicts using two structural analysis methodologies, i.e., Policy Tree and Relational Algebra. Then the results obtained from the test by using an automated tool PolicyVisor, based on the policy tree methodology, are analyzed. It is found from the analysis that even a set of only six rules has number of anomalies. Moreover, it is hard for the human to find such anomalies manually in a larger rule set and failure to find such anomalies leads to change the firewall policy.

Keywords - Firewall; Rule Set; Analysis; Policy Tree; PolicyVisor; Relational Algebra.

I. INTRODUCTION A network system is used for sending emails, making

travelling plans, reading the news, shopping and storing valuable official files etc. In all of these uses protection of information is essential. For large enterprises security includes not only the protection of the critical information of the company but also the security of the entire network.

With the increase in the number of organizations relying on the internet for their businesses it is becoming increasingly important to protect these systems. According to Geng, Flinn, and DcDeourek, firewalls play an important role in securing a network [6]. Almost in every organization firewalls are used as the first line of defense for the protection of its network. Firewalls are recognized as efficient instruments in deploying security in computer networks [13]. When deploying firewalls in an organization

it is essential to verify that the firewalls are configured properly [5]. Configuring the firewall is perhaps the most important phase in ensuring an organization’s security policy. Understanding the outcome of the configuration of a firewall is very hard before firewall is deployed. According to Bandara, Kakas, Lupu, and Russo, to analyze firewall configuration for errors and to verify that they correctly implement security requirements, tools are needed [14]. In this paper, firewall background is described followed by a number of factors that make the configuration difficult. Different approaches for the analysis of firewall configuration are discussed and then two methodologies of structural analysis of firewall rule set configuration are compared to address different problems.

The rest of the paper is organized as follows. Section II briefly explains the firewall rule set. Section III describes the related work. Section IV performs test on a sample firewall rule set using two different methodologies. Section V analyzes the test result found from Section III. Section VI is about the future work and Section VII concludes the paper.

II. FIREWALL RULE SET A common way to keep the attackers from breaking into

the network from outside is to use firewall. A firewall filters each packet coming in and going out and makes sure the source and destination of the packets are trustworthy.

Firewall rule sets consist of rules whereas each rule consists of an action and an associated condition. The action is either accept or deny, whereas associated condition specifies the source, destination IP address, protocol , port number etc. of the packet. To reach a decision concerning a packet, the rules in the sequence are examined one by one until the first rule that’s condition is satisfied by the packet fields, is found.

For most firewalls, the rule set is much larger and detailed. When a packet arrives to the firewall, the firewall will inspect its protocol, the source and destination addresses and ports. Firewall compares the details of the packet against the rules in the rule set, from top to bottom

until a match occurs. Geng, Flinn, and DcDeourek state that “the firewall will execute the action of the first rule matched regardless of any following rules that may match” [6].

Various firewall implementations use different sequence of rules processing. Basically there are two matching strategies [11], single trigger and multi-trigger. Single trigger processing means that an action of the first matching rule will be performed, whereas multi-trigger processing means that all rules will be matched and an action from the last matching rule will be performed.

Configuration is a crucial task, probably the most important factor to determine a security level, stated by Yoon, Chen and Zhang [10]. To configure the firewall the administrator needs to add, remove or change rules in the firewall rule set. These changes in the firewall rule set increases the likelihood of introducing some errors that could make the network vulnerable to some serious attacks.

One reason firewalls are so difficult to manage is that slight differences in the rule set can cause dramatic changes in the behavior of the firewall [7]. A firewall rule set is a list of a large number of rules. Making sense of the rules and especially of the interaction between different rules, poses a challenge.

Wool quantifies the complexity of a rule set in terms of the numbers of rules, network objects and interfaces by analyzing firewall rule set from different enterprises [10]. He further proposes the following formula for the measurement of the firewall rule set complexity: “RC = Rules + Objects + interfaces* (interfaces-1)/2” Where RC represents rule set complexity, rules represent number of rules in the rule set, objects represent the number of network objects and interfaces represent the number of interfaces on the firewall.

III. RELATED WORK Mayer, Wool and Ziskind have proposed an algorithm

that converts the routing table into the network topology file. Based on the algorithm a tool called FA (firewall analyzer) is developed [15]. It tests the validation of the policy before it is actually deployed. It automatically generates queries and on the basis of those queries analyzes the firewall policy. Since the tool automatically generates queries, therefore, it does not fulfill the user requirements.

Gouda and Liu proposed a method called structured firewall design to reduce redundancy in the firewall policies [16]. They use firewall decision diagrams (FDD) to address the problem of completeness, compactness, and consistency among the firewall rules.

To improve the performance of the firewall, Katic and Pale presented rule optimization solution [17]. Based on the method of rule optimization mechanism a tool has been developed called FIRO that works for IP tables Linux firewalls. The optimization procedure is able to remove redundant rules and merge similar rules, however, it is unable to discover other anomalies, e.g., correlation, shadowing etc.

To reduce the size of firewall rule set, two algorithms, i.e., SSO (simple substitutional optimization) and CSO

(complex substitution optimization) have been proposed by Yoon, Chen and Zhang [10]. Using these algorithms a large group of rules is replaced with a small group of rules hence reducing the size of rule set in firewall.

A heuristic sorting technique called sub graph merging (SGM) algorithm is proposed by Tapdiya and Fulp [18]. The algorithm improves the firewall performance by reducing the number of rule comparisons required per packet and re-order the rule set. In addition, the algorithm moves the most frequently matched rules on the top of the rule set. The algorithm, however, does not have the capability to discover anomalies, i.e., redundancy, correlation etc. in the rule set [18].

In this paper, a sample firewall rule set is analysed using two structural analysis methodologies and a tool called PolicyVisor. The results of the analysis from the methodologies and the tool were compared and it was found that some of the anomalies still remain in the rule set after the analysis.

IV. RULE SET METHODOLOGIES TESTING Here two tests are performed to discover possible

anomalies in a sample firewall rule set. Each test uses different methodology, i.e., Policy Tree [2] and Relational Algebra [8] to discover anomalies. Table 1 shows the sample firewall policy on which the test will be carried out.

A. Analysis of Firewall Policy Using Policy Tree In this methodology two rules are selected at a time from

the rule set. Then each field of one rule is compared to the corresponding field of the other rule to find relations between the rules (See [1] for definitions of relations). Then anomalies are discovered on the basis of relations found among the rules. To carry out test the analysis procedure is split into the following three steps:

Table 1. Sample rule set for analysis.

Step 1:

First of all rule 3 and 4 are selected from table 1. So, rule 3 and rule 4 are represented by Rx and Ry, respectively, i.e.,

R3 = TCP 170.0.0.* 174.0.0.7 21 deny R4 = TCP 170.0.0.4 174.0.0.* 21 accept

Order protocol src_ip dst_ip dst_port action

1 TCP 164.0.0.7 178.0.0.* 22 accept

2 TCP 142.186.40.* 160.0.0.30 80 accept

3 TCP 170.0.0.* 174.0.0.7 21 deny

4 TCP 170.0.0.4 174.0.0.* 21 accept

5 TCP 170.0.0.* 174.0.0.* 21 deny

6 TCP Any Any Any deny

Figure 1. State diagram comparing fields of rule-x and rule-y (Diagram taken from [3]).

Now using the state diagram in Figure 1 for rules R3 and R4, it can be seen that after comparison of protocol fields of both rules, state 1 is reached, then source addresses are compared and state 5 is reached, next destination addresses are compared and state 7 is reached. Finally, the action fields of both rules are compared, since both rules have different action values, i.e., deny and accept respectively, therefore, the final state is 14 which shows that the two rules are correlated. Now this criterion satisfies the conditions outlined in the definition of correlation between the two rules. Step 2:

In this step two other rules are selected for comparison, i.e., rule 3 and rule 5 which are represented by Rx and Ry respectively to carry on the procedure on state diagram. Therefore, R3 = TCP 170.0.0.* 174.0.0.7 21 deny R5 = TCP 170.0.0.* 174.0.0.* 21 deny

Now, the state diagram in Figure 1 is used again for the comparison of the two rules to find out any possible relations among the rules. Starting from state 0, comparing the protocol field of both rules, state 1 is reached, because both fields have the same value, i.e., TCP. Now, going further down comparing the source addresses and destination addresses of rules, state 6 and state 9 are reached because source and destination addresses of Ry are supersets for those of rule Rx. Finally the action fields of both rules are compared which will take the situation to state 12 decla-

Protocol

Src_Add

Des_Add Des_Add Des_Add Des_Add Des_Add Des_Add

Des Port Des_Port Des_Port Des Port Des_Port Des_Port

Action Action Action Action Action Action

Rule 1Rule 2

Rule 3 Rule 4 Rule 5 Rule 6

Figure 2. Policy tree for rule set in table 1.

ring Ry redundant to Rx. (According to the definition of redundant rules by Al-Shaer and Hamed [2]). Step 3:

After each rule is compared with the other rule using state diagram, the policy tree is used for the filtering policy. The policy tree in Figure 2 shows filtering policy of table 1 for the easy representation of different relations and anomaly among different rules.

B. Analysis of Firewall Policy Using Relational Algebra This methodology [8] is based on the relational algebra

and raining 2D-box model. Rule 3, 4 and 5 from the sample rule set (see table 1) are considered for the experiment. Step 1:

Initially rule 3 and rule 4 are taken for the test. Figure 3 shows the raining 2D-box model for the representation of rule 3 and rule 4.

Rule 3 =TCP 170.0.0.* 174.0.0.7 21 deny Rule 4 =TCP 170.0.0.4 174.0.0.* 21 accept

According to the definition of correlation (for definitions of relations see [8]), it is evident that rule 3 and 4 are correlated, because some of the packets matching rule 3 are also matching rule 4, while both rules have different actions. This anomaly is also evident from the raining 2D-box model in Figure 3. So, it is clear that rule 3 and 4 are correlated. Step 2:

In this step rule 3 and rule 5 are considered. Both of them are represented by raining 2D-box model in Figure 4.

Rule 3= TCP 170.0.0.* 174.0.0.7 21 deny Rule 5= TCP 170.0.0.* 174.0.0.* 21 deny

According to the definition of redundancy it is evident

that rule 3 is redundant to rule 5 because all the packets that are matching rule 3 are also matching rule 5, in addition, both rules have same actions, i.e., deny. This anomaly is also represented by the 2D-box model in Figure 4. So, it is declared that rule 3 is redundant to rule 5.

Now rule 3, 4 and 5 are combined and represented in raining 2D-box model [8] in Figure 5, which shows the two anomalies, i.e., correlation and redundancy among the 3 rules.

V. ANALYSIS OF TEST RESULTS In previous section, a test was carried out to discover

potential anomalies in a sample filtering policy using two different methodologies for the analysis. Both methodologies use different techniques and procedures but their goal is the same, i.e., conflict detection and conflict resolution. Here the results of those tests are analyzed and their similarities and differences have been discussed.

By applying the procedure of Policy Tree on the sample rule set, two anomalies were discovered, i.e., correlation and redundancy among rule 3 and 4, and rule 4 and 5, respectively. Although in step 2 of Section IV.A, rule 3 is

Figure 3. Correlation anomalies between rules 3 and 4.

Figure 4. Redundancy between rule 3 and 5.

Figure 5. Raining 2D-box model for rule 3, 4, and 5.

declared redundant to rule 5, however, it cannot be removed, because at the same time it is in correlation with rule 4 [1]. Removing rule 3 will change the firewall policy, i.e., it will allow the packets from 170.0.0.* to 174.0.0.7, which are supposed to be blocked.

Also step 2 in Section IV.A declares rule 5 redundant to rule 3 where rule 5 is not involved in any anomaly in the list. It is evident from the policy tree in Figure 2 that rule 3 is in correlation with rule 4 shown by pentagon and rule 5 is redundant to rule 3 shown with the marked square box at the bottom of the tree. So rule-5 can be removed from the rule set [1]. Removing rule 5 does not change the firewall policy; rather it reduces the size of firewall rule set, which is in fact the purpose of the analysis.

Using the same state diagram for rule 1 and rule 2 to compare them with other rules in the rule set, it is concluded that these rules are disjoint and are not involved in any conflicts. Any of the rules in the rule set do not involve in any other anomaly, i.e., shadowing, correlation etc. After removing the redundant rule 5 and keeping rule 1, 2, 3, 4 and 6 as it is, the renewed policy tree looks like the one in Figure 6.

Protocol

Src_Add

Des_Add Des_Add Des_Add Des_Add Des_Add

Des Port Des_Port Des_Port Des Port Des_Port

Action Action Action Action Action

Rule 1 Rule 2 Rule 3 Rule 4 Rule 5

Figure 6. Reduced anomaly free tree.

A. Using PolicyVisor PolicyVisor is an automated tool developed by Al-Shaer,

Hamed, Boutaba and Hasan, for the structural analysis of firewall rule set [3]. Figure 7 shows the step by step details of the policy tree made from table 1 by the PolicyVisor,

whereas Figure 8 shows the result obtained by using PolicyVisor tool for the structural analysis of the sample rule set in table 1. Since PolicyVisor is based on the methodology of policy tree, therefore, the result is almost similar to the one obtained in the test. However, PolicyVisor does not discover the correlation anomaly among rule 3 and rule 4.

It also shows that rule 3 and rule 5 are redundant to rule 6. So it means that rule 3 and 5 should be removed. By removing rule 5 from rule set does not change firewall policy but, by removing rule 3 changes its policy. Because removing rule 3 will make rule 4 to accept those packets which were previously blocked by rule 3 and hence results in the change of policy. The issues related to using PolicyVisor or policy tree methodology can be handled by using relational algebra and raining 2D-box model [8].

Figure 7. PolicyVisor detailed show of constructing policy tree.

Figure 8. PolicyVisor discovers rule set anomalies.

Result from the relational algebra and raining 2D-box model in Section IV.B is similar to the result produced in Section IV.A. Policy Tree compares two rules to discover anomalies, whereas with raining 2D-box model it is possible to compare 3 rules (see Figure 5) and then apply relevant theorem of relational algebra for the removal of anomaly. For example, it is clear from Figure 5 that rule 3 and rule 4 are correlated and at the same time it, shows that rule 3 is redundant to rule 5. Now according to theorem 6 in [8], rule 3 cannot be removed despite the fact that it is redundant, because it is in correlation with rule 4, however, rule 5 can be removed to eliminate the redundancy anomaly from the policy. Using relational algebra for rule 1 and rule 2 in table 1, it is clear that these rules are disjoint and have no relation with other rules in the rule set. Therefore, no anomaly was discovered involving rule 1 and rule 2.

VI. FUTURE WORK In this paper, some factors which may lead to the miss

configuration of firewall, i.e., rule set complexity and number of rules were discussed. The sample rule set of only a single firewall has been considered for the experiment; however, much research has been going on towards the idea of distributed firewalls. In case of distributed firewall it is hard to manage a central policy to control filtering. A limited work has been done on the analysis of distributed firewall policy. More attention needs to be given to properly analyze the distributed firewall policy analysis and management.

Despite the fact that structural analysis of firewall rule set solves many problems with the firewall configuration but there are some errors that can be solved by using active and passive approach of rule set analysis.

VII. CONCLUSION In this paper, three different approaches in which a

network administrator can analyze firewall rule set and verify the firewall security policy are described. These approaches are (a) active analysis (b) passive analysis and (c) structural analysis of firewall rule set. Many tools have been developed; each tool is based on one of these three approaches. Each tool was categorized in its respective category based on the way they analyze the firewall rule set. For example, nmap and Satan are based on active; Lumeta Firewall Analyzer [9] uses passive approach, whereas Firewall Policy Advisor (PolicyVisor) uses structural analysis approach to analyze the firewall rule set. Each approach has its own ideal environment to be used and has its advantages and disadvantages. One single tool needs to be developed that could do active analysis, passive analysis and structural analysis of firewall rule set. Despite more research, there is still more work needs to be done for the analysis of firewall rule set with the said advanced features.

It is proved from the analysis that a slight change in the rule set alters the wanted firewall policy and has a major impact on the firewall configuration.

REFERENCES [1] Al-Shaer E. and Hamed H., “Firewall policy advisor for anomaly detection and rule editing,” The IEEE/TFIP International Symposium on Integrated Network Management Conference USA. pp. 17-30, March 2003. [2] Al-Shaer E. and Hamed H., “Modeling and management of firewall policies,” IEEE Transactions on network and service management, Volume 1-1, pp. 2-10, April 2004. [3] Al-Shaer E., Hamed H., Boutaba R., and Hasan M. “Conflict classification and analysis of distributed firewall policies,” IEEE Journal on selected areas in communications, volume 23, Issue 10, pp. 2069-2084, Oct 2005. [4] Hamed H. and Al-Shaer E., “Discovery of policy anomalies in distributed firewalls,” Proceedings of the IEEE INFOCOM 2004, volume 23, Issue 1., China, March 2004. [5] Eronen P. and Zitting J., “An expert system for analyzing firewall rules”, in proceedings of the 6th Nordic Workshop on Secure IT systems, Pp. 100-107, Denmark. 2001. [6] Geng W., Flinn S., and DcDeourek J., “Usable firewall configuration,” 3rd Annual Conference on Privacy, Security and Trust, Institute of information technology, national research council Canada, 2005. [7] Marmorstein R. and Kearns P., “Firewall analysis with policy-based host Classification,” 20th Large Installation System Administration Conference 2006 (LISA ’06), pp. 41-51, Usenix Association Berkeley, CA, USA. [8] Chomsiri T. and Pornavalai C., “Firewall rules analysis,” Proceedings of the 2006 International Conference on Security and Management, SAM 2006, pp. 213-219, Las Vegas, USA. [9] Wool A., “Architecting the Lumeta Firewall Analyzer,” proceedings of the 10th Usenix Security Symposium 2001 pp. 85-97, Washington DC. [10] Yoon M., Chen S., and Zhang Z., “Reducing the size of rule set in a firewall,” IEEE international Conference on Communication 2007, pp. 1274-1279, Glasgow. [11] Zaliva V., “Firewall Policy Modeling, Analysis and Simulation: a Survey,” [online], Available from: http://www.crocodile.org/lord/fwpolicy.pdf [Accessed: 2nd Jan, 2010]. [12] Wool A., “A Quantitative study of firewall configuration errors,” Computer, vol.37, no. 6, pp. 62-67, June 2004, USA. [13] Ghiran, A.M., Silaghi, G.C., and Tomai N., “Ontology based tools for automating integration and validation of firewall rules,” Proceedings of 12th international conference on Business Information Systems 2009, pp.37-48, Poland. [14] Bandara, A.K., Kakas, A.C., Lupu, E.C., and Russo A., “Using argumentation logic for firewall configuration management,” Proceeding of the 11th IFIP/IEEE International Conference on Symposium on Integrated Network Management, 2009, pp. 180-187. New York, USA. [15] Mayer A., Wool A., and Ziskind E., “Offline firewall analysis”, international journal of information security, vol. 5, n.3, 2006, pp. 125-144, Berlin. [16] Gouda, M.G. and Liu, A.X., “Structured firewall design,” Computer Networks: The International Journal of Computer and Telecommunications Networking 2007, vol.51, no.4, pp. 1106-1120, New York. [17] Katic T. and Pale P., “Optimization of firewall rules,” 29th International Conference on Information Technology Interfaces 2007, pp. 685-690, Cavtat. [18] Tapdiya A. and Fulp Errin W., “Towards optimal firewall rule ordering utilizing directed acyclical graphs,” Proceedings of 18th International Conference on Computer Communications and Networks 2009, pp. 1-6, San Francisco USA.