Measuring and Evaluating the Effectiveness of Information Security

Measuring and Evaluating the Effectiveness of Information SecurityMario Sajko

ABSTRACT

State of information system security is described with the large number of features and indicators for different areas of informationsecurity. Depending on the area of a business in the company and the security objectives, for the effective security management it isnecessary to select appropriate indicators and establish a process of their monitoring and measurement. Systematic application of suchindicators is usually defined as a metric of standardized measures and methods of measurement and interpretation of results. Althoughin the literature we can found examples of such indicators and specific suggestions about how they collect and measure them, thewhole area of the information security measurement is yet unexplored. The focus of this paper is to recapitulate past experience andresults in the systematization and structuring of security indicators which is not completely clarified. The aim is to identify existingexperiences concerning the application of security metrics as an instrument of evaluation and assessment of information systemsecurity.

Key words: security metrics, security performance, information security evaluation

INTRODUCTION

There are numerous reasons for interest in information security. Laws, demands of regulators, need for standardization and marketdemands are some of the sources that are the reasons for increasing interest and responsibility for security. The demands of businesspartners as well as internal demands for a process promotion can be added to all this. With this, the increasing need for security comesout of the increasing vulnerability of information system (particularly because of even bigger complexity of the system,communication intertwinement and using of insecure components and networks). Despite different security management approaches,the security level still remains unknown or uncertain.The experience shows that it is impossible to manage the processes if they cannot be measured. So aside from the type of activity it isnecessary to monitor the security results, and sometimes the level of conformity with standards (usually in the form of norms of thebest practice or rules) for continuous improvement. It is not a question of this paper whether we require measurement and metrics, butwhich measures and metrics we should use to achieve the desired goals. Standards, demands of regulators, laws and examples of thebest practice can present the reference for efficiency comparison and improvement, but the data about proper achievements whencompared in a certain period can also be used as a reference. This is the area of security metrics that will be explained in the furthertext.Today management is mostly interested in answering the following questions:

- Are we more secure than before?- Are we secured enough?- How to compare security level with security demands?

All these questions can be answered by applying metrics or set of tools for monitoring, assessment and measurement and presentationof measuring results. Metrics consist of group of tools connected with achieving particular features and security levels [Deswarte], andin [Swanson] we can find the fact that metrics answer the question how good and appropriate implemented policies and proceduresare. The term of metrics is according to [Vaughn] the transformation and presentation system suitable for making reports tomanagement, and US National Information Systems Security Glossary sees metrics as an instrument for making decisions,documenting and threats counting as well as a part of security risk managing process. Aside from that, metrics consist of the followingelements [Kovacich]:

- object of measurement- referential values- sensors- mechanisms of comparison- initiators of the action- values- ways of presentation- time dimension (measuring or monitoring)

AcademyPublish.org – Risk Assessment and Management 402

On this occasion the definition of metrics will be taken form ISO standards [Humphreys] according to which:- metrics defines a measuring system or standard; measuring scales and units for monitoring efficiency- and the measurement is the act of determining quantity, size of degree (of something) by using standard group of measures

and measuring procedures defined by metrics

Fig.1 is going to serve as the additional explanation of the notion of metrics, according to which metrics can answer on the followingquestions: “What is measured, why it is measured and who measures it?”

Fig.1. The role and application of security metrics [Vaughn]

Experience and works in the area of security metrics

There are numerous results about the security metrics use and its role. It is used for documenting (of security state) and coordinatingof the results that should be corrected [Kormos], for measuring security programs results [Lennon], for understanding and managingthe security demands [Robinson], as a tool for evaluating the security controls results, establishing the size of risk, but also for makingpositive influence on the behavior of employees [Kovacich], [Payne] and [Villarrubia].The notion of metrics is always present in security criteria, standards and concepts. In this context the term of metrics is mostlymentioned as a tool for assessment, evaluation, coordination and alignment with security demands.

For example, ISO/IEC norm states in the edition Code of practice for information security management, about the need for adoptinguniversal and balanced measuring system as a condition for successful implementation of security programs. When we talk about ISOstandard and particularly ISO/IEC 27001 norm, it is directly demanded that information obtained by measuring with metrics has to beincluded in the decision making process about the actions that should be overtaken for security state improvement. Specific directionsabout their use are available in ISO BIP 0074 [Humphreys] instructions that define the ISMS (Information Security ManagementSystem) classes of metrics and measuring.

NIST (National Institute of Standards and Technology) in its publication NIST SP 800-55 "Guide for the specific development,selection, and implementation of IT system-level metrics" [Swanson], defined the way of applying and developing specific securitymetrics for application in different areas of security measuring results. According to CMMI model of maturity assessment,measurements and metrics are a component part of monitoring process of security programs maturity. There are also some othersources of metrics use as e.g. metrics as Common Criteria (CC) demands and it is important to state the results of ISACA instituteconverted into COBIT criteria that integrate the metrics during the evaluation and assessment of process areas features.However, the fact is that the use of metrics is still not sufficient and it is used only on limited number of groups of indicators.Robinson [Robinson] states that the biggest number of data about security program quality refers to metrics for detecting malignantcode, statistics about entrance in the rooms and data about quality of spam messages. And human factor as a factor is often ignored.

The problem is in inadequate metrics standardization and unknown forces about the way of its use in wider context of securitymanagement. The existing group of metrics is also not united in a unique system so it is hardly applied on the whole organization[WISSR] and [Kormos]. So it seems that risk metrics is still “immature” discipline that is explained and understood in different ways.What is also unclear is a question of a group of variables that are used for security measurement, and which features of the system aremeasured, and it is even less known what the measured results present, and the way of their gathering (particularly those factors thatare measured indirectly), quantification of the described indicators etc.

The problems rise from the fact that security state, not only which is described directly by measurable attributes, is often described byabstract indicators. What is also unsettled is the area of basic terminology (metrics, measuring, measured, indicators) and the area(domain) that metrics are related to it. So this is the reason why it is not clear what the metrics describe, how to interpret and assess it.In each case, inconsistent terminology [Lennon] and difficult data gathering for some metrics [Robinson] (e.g. for intellectual proprietyassessment), make the further development of security metrics complicated and slow.

WHAT IS WHY THE WHOMEASURED MEASURE MEASURE = METRICS

TECHNICALPROCESORGANIZATIONSYSTEM

DESCRIPTIONPREDICTIONCOMPARISON

PURPOSE TYPE TARGETAUDIENCE MEASUREMENT FACILITY

TECHNICAL EXPERT

DECISION-MAKERS

EXTERNAL AUTHORITY

× ×


Past contribution of security metrics structuring

More attempts have been overtaken to make security metrics and the way of their application clear. We can find them in the results ofresearches done by foreign corporations and institutes: Corporate Information Security Working Group [CISWG], WISSR, SSE-CMM [Jelen], NIST [Swanson], CSSPAB [Nielsen] and authors Vaughn, Lennon and Seddigh. The contribution which particularlystands out is WISSR workshop during which the problem of information security measurement and the area of measurement werediscussed and some directions were offered. It might be that the best attempt of metrics decomposing (in the form of taxonomy) isexplained in [Vaughn] where the metrics are listed according to the types of measurement and security systems areas. NIST institute[Lennon] suggests the basic metrics division on the metrics for public and private sector (for numerous regulative demands forsecurity public organization security). From other authors it is worth to mention Villarrubia according to who the metrics are dividedregarding the size of organization they are provided for, while [Kormos] puts metrics in groups by using top-down approach (Fig.2)worked out according to instructions of SSE-CMM models.

Fig. 2. An example of "top-down" the division of security metrics

But, in each case the domain of security metrics is not concluded, and the offered security metrics divisions are not universal, despitethe fact that they are a worth starting point and attempt of structuring [Vaughn].

Problems and aims of work

According to the previously mentioned text, the notion of security metrics includes different aspects of security systems. The fact isthat there doesn’t exist a general concept of security metrics nor standardized measures for ranking information security tha t leads tounknown forces in their use and inability of unambiguous evaluation of security efficacy. Or in other words, the question of choosingand using the metrics as well as their use in solving determined security problem categories is not completely explained, and what ismissing is universal decomposition and taxonomy.

In the further part of this paper some of more important metrics will be described (risk evaluation, maturity assessment,„benchmarking“, monitoring of security performances etc), and before that the dimension of their use for information securitymanagement will be studied in next chapters.

DIMENSIONS OF SECURITY METRICS

For the purpose of security metrics structuring we will first explain the basic dimensions of their use. In literature we can findopinions about the areas covered by security metrics. Jonsson differs only metrics for risk measuring, measuring of securitymechanisms features (for certification purposes) and measuring for monitoring unauthorised entrances in information system.According to [Vaughn] the total area of metrics is divided on metrics form the aspect of technical features, managing abilities andassessment of vulnerability.According to NIST [Lennon] interpretation of the areas covering security metrics is a little different (Fig.3). The areas are not clearlydefined but specific basis for security measuring program according to which the basic starting point is a strong management support(metrics for corporative security management monitoring etc.) are defined. The following level of metrics consists of metrics formonitoring the successfulness of operative implementation of security (level of policies and procedures realization). The third level

decreases

vulnerability

acces control

- # Of false report attempts- # Of infections by viruses /

per month- The frequency of audit- The existence of adequate

procedures

- The implementation of IDS- The time elapsed since disclosure of

incident to corrective action- # Of external users that require stronger

passwd- # Of successful penetration sustava

internal external

acces control


consists of quantitative metrics formed for data gathering and their presentation. And finally in the end the last level ensuresmeaningful data for managing, orientation on goals, continuous monitoring and directing towards improvement.

Fig.3. Dimensions of security metrics application

ISO standard measuring differs considering the security areas defined by ISO 27001 standard. According to the manual Measuring theeffectiveness of your ISMS implementations based on ISO/IEC 27001 [Humphrey] we can differ:

- metrics for monitoring the results of managing controls- metrics for monitoring the assessment process and repeated assessment- metrics for monitoring the results of operative controls- metrics for monitoring the results of physical controls- metrics for monitoring the results of technical controls

Considering the mentioned sources, metrics definitely covers technical, organization and operative security dimension (Fig. 4) withhuman factor as a special measuring area contained in responsibility for security.

Fig. 4. Dimensions of security evaluation

Each of the stated dimensions is applied to specific areas of security system (Table 1) and it is characterized by certain way ofmeasuring. Direct, indirect, relative and standardized measuring is dominantly used:

- managing aspect – standardized/indirect measuring (includes measuring of process quality and system coordination)- operative aspect - direct/relative measures (includes measuring of numerable sizes that quantify the process )- technical aspect – standardized /direct measuring (includes measuring physical sizes and numerable units/measures)- human aspect – indirect/relative measuring (includes quality measuring, satisfaction and abstract sizes)

Table 1. Security measuring areas

Measurements fields Fields of applicationmanaging aspect Software support, process maturity, the implementation of the

policy, certificate of successoperative aspect Ability to plan, response to incidents, maintenance, training,

performance securitytechnical aspect Identification, authentication, system records, incident

statistics, penetration testshuman aspect Customer satisfaction, accountability and user knowledge,

human error, training

SUPPORT OF MANAGEMENT

ESTABLISHED PROCESS

QUANTITATIVE MEASUREMENT

EVALUATION OF METRICS


Metrics for a particular area are determined by features of security area. The use of different combinations of objective and subjectiveassessment techniques, qualitative and quantitative measures, static and dynamic measuring, direct and indirect indicators andabsolute and relative sizes [Villarrubia] is possible. Some of them will be the object of observation in the following chapter.

TYPES OF SECURITY METRICS

In the following part the most important metrics for security assessment and evaluation will be presented. This short analysis has thepurpose to point out the previously described problem of the absence of standardized metric division, what is the reason for problemsrising in their use.Some of the most important security metrics are risk metrics. This subgroup of metrics presents the instrument for security systemevaluation, measuring the size of uncertainty that the consequences for business organization will appear because of threats (andweaknesses of security system). They belong to marginal area of measuring the quantitative quality process indicators and technicalaspect of security.

The other group of metrics, maturity metrics of security program, is developing as a separate direction. There are more maturityassessment models that are used in practice as instruments for revising the existing security state. Despite the fact that maturityassessment metrics assess the security features in wide scope, they are more oriented to the measuring of management support andevaluating of process successfulness.

With them there is a bigger group of so called performance metrics composed of metrics for monitoring risk indicators, metrics formonitoring the degree of security program implementation, metrics for monitoring the degree of security goals realization, metrics for„Benchmark“ measuring etc. In the further text they will be described in short.

Security risk metrics

As a part of the information system risk management process, risk assessment was first mentioned in 1974 when the FederalInformation Processing Standard (FIPS) issued the publication Automated Data Processing Physical Security and Risk Management.From then onwards security risk assessment is widely accepted as a first stage and a key component of risk management, which isfundamental in choosing effective security measures. Also, risk assessment is especially useful because determined risk size is anindicator of required security and indicator needed for cost protection management.

Risk metrics is a function consisting of a few factors. NIST model [Lennon] as one of the better known standards, calculates risk onthe basis of 6 factors: threat power, assets value, threat appearance probability, assets sensibility, influence on assets, existing security.There are also some other approaches that combine the risk indicators (Table 2) in different ways.

Table 2. Examples of security metrics

Risk function of NIST method [Swanson] is determined by probability that threat will use the information resource vulnerability, bystrength of threat influence and possibility to decrease or exclude the risk by using security controls. The assessment itself is based ontwo dimensions assessment matrix and qualitative methodology (ranking).

Risk function according to Guide to BS 7799 Risk Assessment and Risk Management [Humphreys] defines risk as asset function,thereat, probability of threat appearance, vulnerability of existing security controls and strength indicators of risk influence. Metriccombines qualitative (information assets) and quantitative (physical assets and software) assessment methodology. The use of one of 4offered techniques is suggested for risk measuring.

Source Risk metricsMickey Krause [threat] × [vulnerability] × [value]FMEA [S]×[O]×[D]CRAMM [value] × [threats] × [vulnerability]RuSecure vulnerability, information value, impact, frequency and

capacity for a risk reductionFRAP vulnerability, business ImpactNIST rang matrixISO assets, threats, likelihood of occurrence of threat,

vulnerability, current protectionOctave [Assets] × [threat] × [vulnerability]COBRA the relative level of compliance with the standardWhat-if subjective assessment of worst-case scenarios


- Matrix with Predefined Values- Ranking of Threats by Measures of Risk- Assessing Value for the Frequency and the possible damage of Risks- Distinction between Tolerable and Intolerable RisksRisk function according to CRAMM methods is mostly qualitative despite the fact that for some categories of business assets(physical) the possibility of quantitative metrics are supported. Qualitative indicators are quantified according to some specific scalesand to give distribution in advance specific for particular risk methods elements.

Risk function according RuSecure methods includes assets vulnerability and values, threats strengths, threat influence it has on assets,possibility of threat activities and frequency of threat activities and all the sizes are expressed in qualitative terms. Qualitativeintensities are transformed into sizes suitable for numeric risk expression by the use of metric scales.

Risk function according to OCTAVE [Alberts] methods sees the risk and a function of information assets and its values that isimportant for business, threats that endanger IS assets and vulnerability that expose the information resources to threats. The basicassessment subjects in this method are immaterial but they can be adjusted to material risk assessment.

Risk function according to FRAP [Peltier] methods combines' qualitative sizes of risk influence on business and vulnerability ofinformation assets by using ranks matrix. On the basis of the obtained intensities and security controls defined in advance, securitymeasures are set up.

What if technique is a graphic analysis that is based on individual or group brainstorming. It consists of the use of structuralquestionnaires about the potential events that can outgrow into accidents or stimulate security problems and establishing the rightsecurity measures according to the established problem. The method is in great deal subjective, and risk is expressed in quantitativeterms

Risk metrics according to FMEA/NASA is calculated as a mathematical function [RPN] depending on the results [S], probability [O]that a particular case will lead to the mistake connected with the defined results and capability to detect the mistake [D] before itrealizes its results. The assessment metric is based on the quantitative risk ranking.

From the enclosed it is obvious that risk metrics components are different and combined in a differently. Thus risk is expressed inqualitative terms (big, prominent or critical) in descriptive terms (“risk can endanger our vital resources”) or in quantitative terms (onthe scale from 1 to 5 or absolute size in some rank). Because of that the assessment results and the form of their presentation(graphical, in tables, numerical) are different. Such differences of risk presentation can completely confuse the decision maker. It is afact that:

- risk factors are combined and assessed in different ways- different risk values over the same group of assessment subject can be obtained by using different methods- relation between the applied metrics and information resources for which metrics are used is not clear

In such situation to risk assessors it is not clear which assessment metric to use, and to decision makers about the investments insecurity is not clear if the assessment results are authentic. That is why some justified doubts in assessment risk results exist and alogic question about which metrics is better is asked.

Metrics for maturity models

There are more assessment and evaluation models for maturity of security programmes out of which the best known are SSE-CMMI,NIST, CERT and COBIT. Metrics are a component part of these models and usually are connected with process areas and evaluationof the degree /level of their implementation. In modern practice the maturity assessment models are used mostly to search the qualityof security managing process and to understand information security better. The maturity models are also a guide for those controlelements that are to be implemented. “Maturity” is in a certain connection with risk so the maturity increase will be reflected on therisk profile of the whole organisation, attitude towards security and better investment return in security.

So maturity assessment is useful and important measure during the assessment of the proper organisation and also when comparingone organisation with another. Although the maturity level can be used for comparing with other organisations (benchmarking) it isnot its purpose. They are mostly used in external evaluation of security program. By analysing the particular model of maturityassessment it was determined that their basis are security criteria that include different aspects of IS/IT managing. Maturity or abilityof organisation to manage its security and resists to security threats is determined by assessing the status of particular criteria.According to criteria and sub criteria structure and as well as their content there different models exist. On the other side intensitydegrees or scales for quality process assessment structure are relatively equal. Comparing analysis of their features is shown in Table3.


Table 3. Examples of security maturity models

It can be presumed that maturity assessment by using different methods on the same sample, will give different results althoughresearches form that field are not known. Mapping the relationship between particular models will exceed the limits and aims of thispaper so it can be the topic of some future researches.

Metrics for evaluating goals and performance indicators

One group of security indicators is about monitoring the realisation of security goals and tasks, security controls efficacy and results,degree of security programme implementation, suitability of conducted procedures and identification of possible improvements. Inliterature such indicators are called with different names, and theoretical study is inconsistent and inadequate. Independentinternational institutions dealing with information security went furthest in their description. (ISO, ISACA, ITCGI).The reason that alittle is written about this theme probably is the fact that such indicators should develop from case to case and cannot be simplycopied from one to another environment. So for the needs of this paper they will be marked with a common denominator –performance indicators. They are a component part of metrics (with other elements) and we can find the opinion that performanceindicators are themselves metrics, but metrics is not always an performance indicator [Koot]. The key difference is that performanceindicators always reflect strategic value while metric can present the values of business process as well. Performance metrics serve ashelp to measuring the achieved improvement according to set tasks and ensure information about the successfulness of securityinitiative implementation.Especial interesting performance indicators are so called key goal indicators (KGI) and key performance indicators (KPI) and also the„benchmark“ measuring. Key goal indicators (KGI – Key Goal Indicators) are used for measuring goals of a process. According toCOBIT Control Objectives, KGI (Key Goal Indicators or outcomes measures) are measures that tell to management if the process hasfulfilled the business demands that are usually shown in the form of criteria. They are the measures “what” has to be realised, and ameasuring indicator of how successfully the process realises the goals.Key performance indicators (KPI - Key Performance Indicators or performance indicators) determine the achievement of the goals ofa process. Some of the areas of KPI monitoring can be:

- adopting the programme about rising security knowledge- required or approved exceptions in relation to risk policy- infection by malignant programme code- unavailability of important IT services- time necessary for instalment of programme corrections- time span between leaving of employees to cancellation of their user ‘s account- discovered areas of wireless approach within the company

In each case the quantitative measures of KPI, arranged in advance, reflect critical factors of successfulness. For example one of theindicators can be the percentage of loss because of the incidents regarding the other losses or even there is a better example of how

Model Maturity model The focus of the modelNIST CSEAT IT 1. policy

2. procedures3. implemented system4. trial5. integration

the quality of documentation

Citigroup’s Info. Sec.Evaluation Model(CITI-ISEM)

1. complacency2. cognition3. integration4. The usual practice5. continuous improvement

the corporate knowledge andlearning in the organization

COBIT 1. Initial / ad hoc2. Repeatable but intuitive3. defined processes4. Managed and measurable5. Optimized

a review of specific procedures

SSE-CMM 1. informal2. Planned and supervised3. well-defined4. quantitatively controlled5. continuous improvement

the engineering and designsecurity

CERT/CSO 1. exist2. repeatable3. assigned responsibility4. documented5. revised and Updated

to measure the quality ofdocumentation


successfully such incidents are solved. Security features should be turned into language understandable to management by using KPI.The examples of KGI and KPI indicators are shown in Table 4.In literature we can find the term of key risk indicators (KRI), but we can rank them among KPI with the indication that their use isexclusively linked to a security risk managing process. KRI should be measured in the context of KPI and their reporting should be acomponent part of the risk managing process.

Table 4. Comparison of KPI / KGI indicators

A special type of security performance measuring is „benchmarking“ measuring (or " benchmarking process") used in in a especiallystrategic management of a business organisation. Such measuring is used to evaluate the aspects of business processes in relation tothe best practice, mostly inside the profession. But benchmark measuring can be applied also to measuring inside the company.Benchmarking can be a unique process event but it is often treated as a continuous process of proper performance comparison.

D. Metrics for calculating investment return in security

In a special group of metrics are the metrics presenting the profitability of invested capital in security (Return of Investment).Generally ROI value is obtained in the way that some of the sizes reflecting the profit are used in numerator (net profit, gross profit,net or gross profit) divided by the amount of the invested capital [Leach]. While investment in information security can be shown infinancial terms, the investment profit because of hardly measurable financial profit should be shown in different way.

The problem can be solved by putting in relation the security controls costs (that are mostly easily measurable) and costs (or losses)appearing because of inappropriate security level (e.g. data renewal, process break, compensation for caused losses). Risk size (thatrepresent the level of uncertainty and potential loss) is used to forecast such costs, and function ROSI Return of Security Investment[Sonnenreich], is used to calculate the return of investment on the basis of decreasing risk exposure (Eq. 1):

urityininvestmentosureriskROSI

sec__

exp (1)

For quantifying risk exposure there is not a standard method, but the use of ALE method is very common (Annual Loss Expectancy)which combines the size of potential loss and probability of loss and the given sum is shown in financial terms. The components ofrisk metrics of ALE methods are (Eq. 2):

- influence of manifestation in financial value I(Oi) – intensity rank 0 - n (n is not limited )- frequency of manifestation of Fi – is expressed as numerical size of relative frequency (threat) in a period of one year.

ALE = i

n

ii FOIALE )(

1

(2)

Establishing ALE size consists of three basic steps:- determining the potential single loss ( proprieties value × exposure factor = single risk exposure)- determining the threats for information proprieties (loss probabilities)- combining sizes of potential loss and loss probability

The deficiency of that approach is aggravated establishing of the risk exposure for non-material components of information system.

KGI KPI-% Of projects completed on time-% Change in the system completedwithin the required time- # Of projects where the plannedtargets were not achieved due to poordesign applications-% Of users satisfied withfunctionality of delivered items

- # Of repeated incidents- # Of incidents resolved with remoteintervention- # incident resolved after the estimated time toresolve them- # Of registered incidents (by Service Desk) byCategory- Average time to resolve incidents by category-% Of incidents resolved after the first call (toService Desk)


CONCLUSIONS

This paper describes a bigger number of metrics for evaluating the efficacy of information security system. The choice of metrics thatwill satisfy the need for examination and quantifying of the process depends on many factors that should be examined (as for examplethe necessary precision and level of the detail that should be discovered by measuring or capability to transform data about differentsecurity criteria). In any case the metrics suitability indicators are:

- ability to ensure objective measuring- Co-ordination with security policy- indicating the trend and make the statistical analysis possible- fulfillment of the security criteria demands

Although it seems that the area of information security metrics is defined, and the use of metrics for monitoring and evaluatingsecurity is clear, many thing are still incomplete. Formalization on the field of the evaluating and measuring different securityindicators is still missing. It is necessary to form the domain structure and hierarchy on the basis of which the whole area could betransferred into unique model.

REFERENCES

Alberts Christopher, Audrey J. Dorofee, USA(2009), OCTAVE Method, Volume 1-6, CMU, www.cert.org/octave

CISWG, USA (2005). “Report of the Best Practies and Metrics Teams”, Corporate Information Security Working Group, Goverment ReformCommittee, http://www.educause.edu/ir/library/pdf/CSD3661.pdf

COBIT, USA (2007), COBIT 4.1, IT Governance Institute, ISACA.

CRAMM Management Guide, Crown, <http://www.insight.co.uk/newsarchive.htm>,

Deswarte Y., et. All., (2004). “Experimental Validation of a Security Metrics”, Centre National de la Recherce Scientifique,.http://philby.ucsd.edu/~cse291 _IDVA/papers/rating- position/Deswarte.pdf

FMEA Methodology, USA (2009), Kinetic LLC, http://www.fmeca.com/

Humphreys Ted, Plate Angelika, UK (2006). “Measuring the effectiveness of your ISMS implementations based on ISO/IEC 27001”, BSI.

Jelen G., SSE-CMM Security Metrics, ISSEA, http://www.sse-cmm.org/metric/metric.asp

Jonsson Erland, “An Integrated Framework for Security and Dependability, Chalmers University of Technology”,http://www.windowsecurity.com/uplarticle/5/Paradigms-nspw98-print.rev0001.fm55.pdf

Koot Matthijs,(2006). “Towards KPIs for Enterprise Security Governance, University of Amsterdam”,https://alumni.os3.nl/~mrkoot/courses/ ICP/ICP-paper_metrics.pdf

Kormos C., Using Security Metrics to Assess Risk management Capabilities, National Security Agency,http://csrc.nist.gov/nissc/1999/proceeding/papers/p29.pdf

Kovacich Gerald, USA (1997).” Information Systems Security Metrics Management”, Computers & Security, Vol. 16 , pp. 610-618.

Lennon B. Elizabeth, USA (2003), “ IT Security Metrics, Information Technology Laborator”, NIST, Information Security News: ITLBulletin for August 2003

Leach John, “Security engineering and security RoI”, John Leach Information, Security Ltd,<http://www.compseconline.com/free_articles/cose_22_6.pdf >

Nielsen F., (2000),”Approaches to security metrics”, CSSPAB Workshop on "Approaches to measuring security,http://www.acsac.org/measurement/ proceedings/wisssr1-proceedings.pdf

Noy, N. F. and McGuinness, D. L. (2000), “Ontology Development 101: A Guide to Creating Your First Ontology”, http://smi-web.stanford.edu /pubs/SMI_Abstracts/SMI-2001-0880.html,

Payne C. Shirley USA (2002). “A Guide to Security Metrics”, SANS Institute, http://www.sans.org/rr/whitepapers/auditing/55.php


Peltier R. Thomas, USA(2000), “ Information Security Risk Analysis”, CRC Press LLC, Boca Raton, Florida

Robinson Chad, USA (2004), “Collecting Effective Security Metrics”, Robert Frances Group,http://www.csoonline.com/article/219182/collecting-effective-security-metrics

RUSecure, Information Security Officer's Manual - the ISO Manual, Glendalesystems, www.rusecure.co.uk

Solms B., USA(2001), “Information Security - A Multidimensional Discipline”, Computers & Security, Vol. 20,http://www.sciencedirect.com>

Solms B., USA (2000), “Information Security-The Third Wave”, Elsevier Scince, http://www.sciencedirect.com

Seddigh Nabil (2004). “Current trends and advances in Information Assurance Metrics”, PST, Conference Proceedings,http://dev.hil.unb.ca/Texts/PST/pdf/seddigh.pdf

Sonnenreich Wes, Return On Security Investment. “A Practical Quantitative Model, SageSecure” , LLC, http://www.sagesecure.com

Swanson Marianne, et. All., USA (2003). “Security Metrics Guide for Information Technology Systems”, NIST Special Publication 800-55

Vaughn Rayford B., Jr, (2002). “Information Assurance Measures and Metrics - State of Practice and Proposed Taxonomy”, Proceedings ofthe 36th Hawaii International Conference on System Sciences

Villarrubia Carlos, USA (2004). “Analysis of ISO/IEC 17799:2000 to be used in Security Metrics, Security and Management” pp 109-117

WISSR,USA (2002). ” Workshop on Information Security System Scoring and Ranking, Applied Computer SecurityAssociates,<http://www.acsac.org/measurement/proceedings /wisssr1- proceedings.pdf


Measuring and Evaluating the Effectiveness of Information Security

Documents

Transcript of Measuring and Evaluating the Effectiveness of Information Security