1

ACHIEVING LASTING CHANGE IN MULTI-ORGANISATIONAL TASKS: the case of flood warnings in Australia

John Handmer[1]Flood Hazard Research Centre

Middlesex UniversityLondon EN3 4SF

Chas KeysState Emergency Service

WollongongNew South Wales

Jim ElliottBureau of Meteorology

Melbourne

1 Correspondence to John Handmer, Flood Hazard Research Centre, Middlesex University,Queensway, Enfield, EN3 4SF, UK. Internet: <J.Handmer@mdx.ac.uk>. Fax: +44 181 362 5403.

2

ACHIEVING LASTING CHANGE IN MULTI-ORGANISATIONAL TASKS: the case of flood warnings in Australia

ABSTRACT

Guidelines for achieving change in the performance of a task are of little value unless they areactually adopted and used by both planning and operational staff. This suggests that success - orrather lasting improvement - is dependent on appropriate processes for guideline development. Theprocesses would need to build broad based support for lasting cultural or institutional change. Theauthors were involved in a recent attempt to change the approach to flood warnings in Australia,through drawing up a set of "best practice" guidelines. This paper explores the background tochange, the institutional context, the process of achieving agreement on, and building commitmentfor, the details of the changes, and the need for further continuous effort to ensure implementationof the changes on the ground.

KEYWORDS: Organisational change; flood warnings; warning systems; planning process.

ACHIEVING EFFECTIVE FLOOD WARNING SYSTEMS

Improving warning systems has long been high on the agenda of Australian flood managers. The challenge is to develop processes for the establishment, evolution and maintenance ofsystems capable of delivering reliable, timely and effective flood warnings, which elicitappropriate response from those at risk. But until comparatively recently the most significantchanges were in the areas of flood detection and monitoring, where the application of moderntechnology has led to substantial increases in reliability and in some cases to warning lead time. Although very important, such changes are relatively easy to implement and do not bythemselves give better results in terms of the overall goal of warning systems - which isimproved safety for those at risk and reduced flood damages. A reasonable generalisation is thatthe dramatic improvements in information technology (IT) have yet to play a role in gettingmessages directly to the public at risk, or to making messages more persuasive.

Achieving the necessary changes may pose both conceptual and practical challenges for the manyorganisations involved. Conceptually, organisations need to broaden their perspectives so thatthey ensure that their contributions focus on the overall objective of the task rather than simplytheir own component of it. Furthermore, the warning system itself needs to be defined toinclude those at risk and the process of review and learning, while in practical terms, operationalinteraction between the various organisations needs to become "seamless". Both the policy andoperational levels of the organisations involved need to be (or to become) self critical and to

3

build relationships with other warning system stakeholders. This is made more difficult whenvolunteer organisations, such as the territory and state emergency services (T/SES) occupy acentral position, as they do in Australian emergency management. These groups need to workclosely with the technical bodies providing the predictions and with other organisations such asthe police and media.

An extensive literature exists to assist those grappling with improving flood warnings. It fallsloosely within the disciplines of geography, sociology, psychology and hydro-meteorology, anddeals in depth with most aspects of flood detection, prediction, message communication andresponse. For example, Williams (1964), Foster (1980), Handmer and Ord (1987), EMA (1995)among others, have conceptualised the overall system. Penning-Rowsell and Handmer (1986)have examined the difficulties involved in warning assessment, while numerous studies haveexamined the factors important in warning comprehension and response. Reviews of thisliterature are found in Mileti (1975), Drabek (1986), Handmer and Ord (1987), Burkhart (1991)and many others. National and international reviews of practice and system design are containedin Handmer (1997), Parker et al (1995), Gruntfest (1997), Gruntfest and Huber (1989) and inSmith and Handmer (1987), among others. All this is in addition to the largely separate and vastcollections of literature covering the general fields of communication, persuasion, andenvironmental monitoring and flood detection systems.

Following flooding the performance of many individual warning systems has been assessed -usually in an ad hoc way in that each case is treated differently and past attempts to use standardcriteria for assessment have been very crude. These assessments often find fault with thedecision-making within the system under examination and with all aspects of warningdissemination. Occasionally, flood detection and prediction will also be found wanting. Drawing on the literature mentioned above, these reports generally make perfectly reasonable,but largely predictable recommendations concerning: improved liaison between the variousorganisations involved in the warning process and the need for better working relations;improved and more realistic emergency plans including the use of scenarios; the need to learnfrom the mistakes of the past; the desirability of public education; the potential role of IT;warnings which use language those at risk understand and the need to make dissemination fastand reliable. In short, to incorporate long established disaster related research findings into theirwarning systems and emergency plans while taking account of new trends and localcircumstances. If the evaluation has been undertaken by those from a technocratic background,then instead of many of the above points or in addition to them, flood detection and prediction,hardware, software, and training will feature. Exceptions include some of the assessments ofwarning systems in France by de Vanssay (1992), where attention has been devoted to the issuesof state legitimacy and the deeply held beliefs of key officials.

Like the main findings set out in the flood warning and broader disaster literature,These typicalcategories of recommendations are quite sound - but key areas are missed. In summary, theyconcentrate on the formal or official warning system, with an emphasis on technical aspects andon the attributes which influence comprehension of and effective response to official warnings. There is relatively little on informal warning systems or networks (Parker and Handmer, 1997),or on questions of overall system and more broadly institutional design and change.

Guidelines or manuals of practice for warning systems and emergency management, based on

4

this literature, abound (for example, Drabek and Hoetmer, 1991; US National Weather Service,1997; UK Emergency Planning Society, 1998). They are easy enough to produce, but - as withthe warning system evaluations mentioned above - are of little value unless enthusiasticallyadopted and used by both planning and operational staff. This suggests that success, in terms ofputting in place substantial improvements to warning systems, requires more than a sounddocument. The content of the guidelines - the document - is very important. It is also relativelyeasy to construct in a technical sense as the general principles are well established and accessible. What is much more difficult is to ensure that the development of guidelines captures thedetailed requirements of those expected to implement them, and that operational staff havegenuine ownership of and vested interest in the final document. These points are essential togive the document, and more importantly the ideas it contains, the credibility and legitimacyneeded to make its use natural among the practitioners responsible for flood warnings.

Remarkably, although there is much literature on what should be done and what happens ordoes not happen when warnings are issued, we could find no material on how to convert theseideas and research results into practice on the ground: rarely if ever are the responsible officialsgiven advice on how they can put the good ideas into practice. There appears to be little ornothing on how to get the multi-organisational change and commitment necessary for a wellintegrated and effective flood warning system. This is largely a question of an appropriateprocess.

The production of well supported guidelines should not be seen as the end of the process eitheras the institutional context needs to create a learning environment for continuing evolution andimprovement. For convenience, we can identify four components to achieving lasting changeand improvement in flood warning systems:

1. a recognition at the institutional level that change in the way warning systems areconceptualised is necessary and must be accommodated;2.the development of a cooperative process to develop the guidelines for change, toensure ownership of the guidelines, and to start the change;3. the document containing the guidelines; and4. a process to ensure operational acceptance and continuing evolution.

These issues are examined using a recent attempt to change the approach to flood warnings inAustralia, in which the authors played key roles. Change proceeded through reconceptualisingthe flood warning system, drawing up a set of "best practice" guidelines (Flood warning: anAustralian guide (the Guide), EMA, 1995) - concurrently with starting changes - having theseguidelines accepted as the reference document, and then implemented. This paper explores thebackground to change and the evolving institutional context, the aims of warning system change,the process of achieving agreement for the changes, the need for further continuous effort toensure implementation of the changes on the ground, and the challenge of developingperformance indicators to help ensure continuous review and improvement.

THE FLOOD HAZARD CONTEXT

Flooding and flood warning times in Australia ranges from the urban drainage flash flooding,experienced with little ot no warning in most built-up areas, through to the sluggish inland

5

Murray-Darling river system where a floodwave may take months to travel from the headwatersin Queensland to low lying towns over 1500 kms away. The tropical north is subject to floodsinduced by hurricanes (tropical cyclones) for which their is usually several days warning. Alongthe east and west coasts, rivers routinely inundate or threaten settlements with between severalhours and days of warning time - the same comment applies to the island state of Tasmania. Storms in the semi arid and arid areas of the inland and west result in flash flooding which maythreaten both local people and travellers. All warning systems could be improved, although interms of economic damage and people affected, urban streams and coastal rivers deserve themost attention.

THE INSTITUTIONAL CONTEXT

In Australia, responsibility for flood warnings and response is divided between the nationalgovernment and the eight states and territories. In addition, the private sector is involvedthrough the important role of mass media for warning disemination. The Bureau ofMeteorology is the peak body with national responsibility for flood forecasting. Responsibilityfor emergency response lies with the states and territories. Precise arrangements vary byjurisdiction, but the emergency services and police play key roles in adding locally relevantmaterial to warnings, ensuring dissemination and encouraging appropriate response. Thenational agencies, the Bureau and Emergency Management Australia (EMA), can onlyrecommend changes to these activities through training, publicity and so on.

Emergency management has been undergoing a period of significant change in Australia overthe past decade. Traditionally, agencies with responsibilities for managing hazards - fire brigades,state emergency service (civil protection) organisations and so on - tended to focus on "at thetime" response activities including, in the flood context, the tasks relating to rescue, evacuationand resupply. Preparedness activities, except for those relating to rescue training, were givenlittle attention, and formal planning was rudimentary where it existed at all. The changes soughtin recent times address these deficiencies, and are putting into place a culture more open to ideas,learning and planning. Major change is not new; the federal agency abandoned its focus - interms of both the rhetoric and the reality of funding - on war preparations in the early 1970s,long before either the United States or UK.

Emergency management organisations have generally accepted the "comprehensive" (the fourelements of prevention (or mitigation), preparedness, response and recovery) and "all agencies"approaches as basic pillars in the development of arrangements for dealing with emergencies. These approaches, and the other elements of the Commonwealth Counter Disaster Concepts andPrinciples document (NDO, 1989), have been well established in the disaster managementliterature for some time, but effective implementation has proven elusive - although key elementsof the handbook are being incorporated into legislation in some states; in effect making itsrecommended approaches mandatory. Now, there are moves towards a broaderconceptualisation and standardisation through the use of the Australian/New Zealand RiskManagement Standard (Salter 1996) - which is likely to form the basis of a new internationalstandard for a risk based approach to emergency management. All these changes have the effectof broadening agency definitions of their management roles; definitions which are now widelyseen as having been too limited for agency briefs to be discharged effectively.

6

Planning: emphasis on process

The focus on planning, especially when defined as an iterative process rather than simply theproduction of a document, is particularly relevant to our discussion here. A failure to viewplanning as more than the production of paper plans has long bedeviled emergencymanagement, and other fields where planning has been viewed as little more than a necessaryevil. The literature from the discipline of planning makes is clear that it is a means of achievingsome end - it is not an end in itself (Branch, 1990); or put another way planning is a way ofmaking an idea work (Herrington, 1989). Summarising the often conflicting work on definingplanning, Alexander (1986: 43) writes that planning "must be societal [ie not an individualactivity], future oriented, nonroutinised, deliberate [not trial and error], strategic, and linked toaction." Those undertaking planning need legitimacy as they are planning on behalf of others,they need influence and they need the cooperation of the people and organisations who willmake the plan possible. Planning takes place via an iterative process for which there are manymodels. But all models see "planning as a sequential, multistaged process in which many of thephases are linked to their predecessors by feedback loops." (Alexander, 1986: 44). Feedbackincludes evaluation and improvement.

The Australian national Community emergency planning guide (NDO, 1992) takes account of theessence of this work. Here are two of its statements on planning:

"The planning process allows agreements to be reached between people andorganisations. The written plan becomes a formal record of commitments made."(p2).

"Planning is a continuous process. The written plan is a living document, constantlybeing reviewed and updated." (p5).

One aim of the planning process is to build a constituency of support for the plan or guidelines. The document (called the plan or guide) should be seen as a record of agreements reachedduring the process of planning, but with circumstances constantly changing the document isunlikely to ever be completely up to date. The network of contacts built up through developingthe plan will be able to deal with this, as emergency managers tend to turn to people they knowand trust - professional and personal contacts - for advice and assistance rather than to facelessnames or organisations specified in a document (eg Drabek, 1987; for commercial organisationssee Krackhardt and Hanson, 1993). The emphasis on multi-organisational (or stakeholder)committees in the Australian emergency planning system is an attempt to come to terms withthese issues through strengthening emergency networks and building cooperative linkages. Inthis sense, "planning" is seen as far more important than the production of a "plan".

Flood Warning Consultative Committees: encouraging cooperation

In Australia, an important factor in encouraging planning and cooperation among the variousorganisations involved in flood warning systems - and therefore in moving towards the "totalwarning system concept" - was the establishment of Flood Warning Consultative Committees

7

(FWCCs) in 1988-90 in each state and territory. These were set up by the Commonwealth(national) Bureau of Meteorology. Their purpose was to guide the expansion of the Bureau'swarning system with advice from agencies with potential data inputs and from agenciesrepresenting users of the service. The members, in addition to the Bureau, were generally waterauthorities, emergency service organisations and local government bodies. The committees'business centred on prioritising investments in flood forecasting, where new gauging systemswere being introduced or manual gauges were being replaced with telemetry. The focus was onimproving the accuracy and timeliness of the flood forecasts coming from the flood warningcentres operated by the Bureau in each state.

A recent evaluation showed that flood related agencies welcomed the work of the FWCCs (BoM,1995). It was found that the committees have encouraged a greater level of cooperation betweenagencies with roles related to the flood warning task. The FWCCs have provided a new forumand have achieved more than simply advising the Bureau about its investments. In essence thecommittees have helped build an understanding of the multi-agency nature of flood warning andhave educated personnel about their roles in the warning process. It is unclear that this couldhave been achieved by other means.

THE TOTAL FLOOD WARNING SYSTEM

The institutional evolution, manifest in committees of stakeholders and an emphasis on process,has had the major effect of making the emergency planning system (in so far as it relates to theflood warning role) more open to change and learning (but that is not to say that it could notbecome much more open). The possibility emerged of major change from a somewhatfragmented approach to the integrating concept of a "total flood warning system". TheAustralian Guide has the following definition (p5):

"A total flood warning system integrates flood prediction, the assessment of likely floodeffects, the dissemination of warning information, the response of agencies and thepublic in the threatened community, and review and improvement. These componentsmust operate together for sound flood warning performance to be achieved."

A total system is much more than a list of components, even if the list is somewhat longer thanconventional for flood warning systems. Of course each component must function well - ideallyeach would draw on state of the art knowledge - but as implied in the definition above, of greaterimportance is how they function together to serve the purpose of the system: "to empowerindividuals and communities to respond appropriately to a threat in order to reduce the risk ofdeath, injury, property loss and damage." (BoM & AEMI, 1993). Critical concepts are those ofintegration, cooperation, shared responsibility, and thinking broadly about the problems. Inpractical terms these translate into the need for planning and system maintenance andimprovement through the incorporation of all relevant organisations, including those involvedwith floodplain management as well as emergency arrangements.

Many of these matters, if not all, involve ongoing activities characterised by liaison withcommunities at risk and other organisations in the flood warning system. Self criticism andreflection are also required.

8

THE PROCESS: DRAWING UP THE GUIDE AND ENSURING OWNERSHIP

The process leading to production of the Guide for designing and running a total warning systemtook three years, and as we explain below this is definitely not the end point: making thenecessary cultural changes at the operational end may take another few years (Table 1).

The major challenge was to draw up a document with substance seen as relevant and credible bya wide range of organisations. Given that it is an important principle of Australian emergencyplanning that the contents of such documents emerge from the iterative process of consultationand endorsement described below, the Guide could not be developed in isolation and simplypresented for approval. Even if approval is obtained retrospectively, it is unlikely that any feelingof ownership or commitment to the plan or guidelines would follow.

A national consensus on the need for a guide

In April 1990, the three mainland states of eastern Australia experienced very severe flooding. Twocountry towns (Charleville, with 3,200 people, and Nyngan, with 2,500 people, in the states ofQueensland and New South Wales respectively) had to be completely evacuated and there weresubstantial evacuations from small urban centres in the Gippsland area of the state of Victoria aswell. Nyngan in particular was a major media and political event. Virtually the whole communitywas involved in placing over 200,000 sandbags to heighten the existing levee which created a dry"island" on the vast flooded western plains. Eventually the augmented levee was overpowered, thetown was flooded and the population evacuated by helicopter. The extensive and damaging floodingin the three states put warnings and emergency management under scrutiny.

As eastern Australia was drying out, a national conference was convened to consider what lessonscould be gleaned from the 1990 floods. The conference concluded that warning practices in manyareas were inadequate. Flood predictions had been the subject of strong criticism, and in placesthere was evidence that people at risk from flooding had either not been warned or had notunderstood the warnings provided. A review of the performance of the flood forecasting systemover this period was undertaken by the Bureau of Meteorology. This review showed that forecastrequirements had been fully defined at only 15 percent of the forecast points, indicating a lack ofeffort by emergency management and local agencies in planning their responses to flood forecasts.

As a result of the inadequacies in warning system performance, a national workshop on floodwarnings was convened by Emergency Management Australia in late 1991, with 50 participants fromgovernment (the state and territory FWCCs), and non-government (media and research)organisations involved in various aspects of the warning task from flood detection and predictionthrough to the delivery of warning messages. This workshop ended with a consensus calling for theproduction of a national "guide to best practice" in the field of flood warning. The Guide waspublished in 1995 (EMA, 1995). The aim was to develop a clearer understanding of the floodwarning task amongst practitioners by defining best practice in the field and indicating how it can beapproached in different environments. Crucial to this is a comprehension of the notion of a "totalflood warning system" into which the myriad tasks carried out by a number of organisations must be

9

fitted.

Armed with knowledge of the flood warning role and guidance provided by the workshop, asubgroup of the working party which planned the 1991 meeting took on the task of drafting ahandbook of best practice. In this they were given some resources for meetings by EMA; and theyretained the backing of the participants at the 1991 workshop. Although the establishment of astable team to guide the process, collate material and draft the guidelines was informal andevolutionary, it enabled completion of the task and ensured far broader ownership than would bepossible had one individual managed the process. The "team" represented the Bureau ofMeteorology, the state emergency services, Emergency Management Australia, and research interests. The practitioners were senior people with the unambiguous backing of their organisations. Thegroup had a reasonable spread in terms of experience and types of stakeholders as well as balancingpractitioner interests with critical outside input. One member of the group took on the role ofinformal "leader" which ensured that every now and then the material would be driven to the top ofthe contributors' individual priority lists.

TABLE 1 about here

A long iterative process

Document production was a long iterative process among members of the writing group and theirpersonal networks; and between them and some of the original workshop participants, as well as aloose reference group of people in the broader warning and emergency management industry. Allthis helped maintain wider group interest and commitment, kept the FWCCs involved and helped tokeep the style appropriate given the nature of the guidelines' audience. The writing level proved tobe a difficult issue. The audience was to be practitioners, including volunteer SES staff, police andlocal government officials, however the writing team consisted of two members with PhDs, onewho was studying for a PhD and a fourth who has had a long career near the peak of a scientificorganisation. Once a draft of much of the document was produced, it was also circulated to a fewnon-specialists for comment on writing style and level. They returned a verdict that some sectionswere difficult for a non-specialist to comprehend and could create problems at practitioner level. Combined with other comments, this led to some rethinking. Consideration was given to engagingthe services of a professional editor to rewrite the material at the right level, but for a range ofreasons the team members opted to continue the work themselves. They tried to make the textmore user-friendly and easier to read, and made a decision to insert summaries of the key points ineach chapter.

Eventually, after further detailed review by the informal extended reference group, it was felt that thedocument was complete. It is worth observing that at this stage - the near final draft - many peopletook a very careful look at the document and made numerous suggestions for change! These wereincluded, in some cases after discussion with those advocating changes, and after a final round by thewriting group, the guidelines were sent formally to all FWCCs to seek their endorsement.

The draft document was endorsed by all FWCCs and by EMA. Work did not stop here, and theteam continued to seek high level endorsement from the relevant organisations. In addition,

10

although the team had engaged with a reasonably large number of people throughout the longprocess of document preparation, most of the volunteer "rank and file" members (numbering in thetens of thousands) of the relevant state and territory emergency planning and response organisationswould have been largely unaware of the process. The next stage therefore was to have the materialaccepted by operational staff - as explained below.

THE DOCUMENT

The contents of the document resulting from this process reflect the discussion in the ealier sectionof this paper entitled "The total flood warning system". The Guide begins with identification of thegoals of flood warning. It consciously develops the idea of integration in warning systems beforespelling each element out in detail. Core content chapters deal with: best practice in flood prediction;the means by which these predictions can be interpreted and applied in the local environments offlood prone communities; how warning messages can be designed and communicated; and how thesystem and its vital interactions can be maintained and improved through continuous review. Table2 sets out the Guide's structure and contents.

Also important is the notion that practitioners need to plan their approach to the task beforehand sothat they are able to apply the most appropriate warning strategies for their own area given thenature of the flood hazard there (for example, speed of flood onset), the nature of the community(demographics, in the broad sense), and how the warning task interfaces with local resources andattributes. Planning and maintaining the system are time consuming tasks, which should start withknowledge of the community concerned.

There are also general principles to be inculcated such as the principle that warnings should addvalue to (ie be more than merely) height predictions, provide advice consonant with the nature of thethreat and its likely impacts, communicate using the language of the intended audience, use anappropriate range of techniques, etc. While some of these elements involve investments of time andmoney, others are non-technical and not costly to implement - for example the identification of tasksand procedures. Further discussion of the Guide's contents is found below under "Achievingcommitment at the operational level".

TABLE 2 about here

HIGH LEVEL SUPPORT FOR THE GUIDE

Production of the Guide and its widespread acceptance by FWCC members are important factors infull implementation, but do not guarantee that the Guide will find its way into practice, or that thenecessary changes will become permanent. There needs to be institutional change at the policy level. And it is just as important that operational staff support the Guide and want to implement it on theground. Unless support exists at both levels, the Guide will not become part of normal practice.

Even though the recent review of the FWCCs, mentioned above, found widespread satisfaction, itwas suggested that the committees' terms of reference be broadened to "cover the performance ofall aspects of flood warning procedures". (BoM, 1995: v). This change has been accepted by the

11

Bureau of Meteorology. This takes the FWCCs well beyond the role of merely buttressing thepredictive process - which is what the Bureau originally set them up to do. Following this review,the Director General of EMA has reached agreement with the Director of the Bureau ofMeteorology that the new role of the FWCCs should include encouraging the notion of "bestpractice" and monitoring progress towards higher quality flood warning practice; using the Guide asthe basis.

Central to this change has been the support of the relevant Chief Executive Officers in theemergency services and Bureau of Meteorology, at both federal and state levels. Operationalimplementation of the Guide must be handled in a way which matches the needs and arrangement ofindividual states and territories. All FWCCs endorsed the Guide and six of the eight states andterritories were proactive in publicising it (see next section). Of the remaining two, one has sinceadopted the Guide, while the other state feels that they are already following best practice. As thenational bodies, the Bureau and EMA, and relevant state organisations were involved throughout theinception and development of the Guide, on the whole they welcomed it as their own. It was not aquestion of state reaction to an imposed document, but of implementation by operational staff.

ACHIEVING COMMITMENT AT THE OPERATIONAL LEVEL

It is just as important that operational staff support the Guide and want to implement it on theground: eventually it should become part of normal practice. To help this happen, resources weremade available by EMA to hold workshops around Australia to explain and discuss the Guide. Thesehave been half-day workshops, broadly based as regards agency participation and involving theBureau of Meteorology, police, local government, state emergency service staff and volunteers. Theyinvolved walking through the Guide and the setting of small group exercises tailored to the specificlocal context. The exercises were designed to focus minds on how best to provide warnings in thelocal area. More of these workshops are needed, and that will be the job of the FWCC/SES in eachstate and territory. The seed has been sown but it needs continuing care, including "watering" atpost-flood debriefs, between-flood exercises and regular conferences of SES operatives.

Using the Guide, existing flood warning practice is reviewed and then, for different levels of flooding,the local SES staff are asked to consider who needs to be warned, what information they wouldneed, and how to get it across. This gets points across about different message styles, contents anddelivery modes for different flood severities and different clienteles. Local flood information held bythe SES is used to generate realistic scenarios, which are used as a basis for designing user-friendly,information-rich warning messages for broadcasting on radio. This is all about breaking the oldwarning mindsets and getting SES volunteers to focus on using tools such as "flood intelligencecards" (records setting out the precise local impacts of floods of different magnitudes), appropriatedissemination modes, layered warnings, etc. The intention is to create a new mindset about the taskand its delivery, and to encourage people to break away from the minimalist approaches of the past(for example, that warnings are only about "postboxing" the Bureau's height and time predictionswithout adding value by identifying local inpacts and appropriate actions).

Embedding local change

12

Much of what is done to create high quality flood warnings must be done at the local level - that is,within the communities which experience flooding. Those involved are typically volunteers, whosometimes have considerable local knowledge, but may lack technical expertise in the application ofsuch knowledge to the warning task. It is these people who have to be convinced that the conceptof the total flood warning system is appropriate. Perhaps more difficult is acceptance of the realitythat improvement and refinement is a constant never-ending process as expectations rise, prioritieschange, technology becomes more sophisticated and so on.

It is important to have people in the flood management agencies in these communities whounderstand the warning task and, even more importantly, are committed to it. This task is likely torequire deliberate teaching about the principles of warning as contained in the Guide (see "The TotalFlood Warning System" and "The Document", above). Such "champions" may need to bepurposefully cultivated and developed and kept interested in the task during the sometimes longperiods between actual flood events. Much can be achieved in terms of local learning once a floodwarning has been issued, provided the warning time is long enough - and provided that the basicorganisational structure is in place which enables experienced professionals to assist and teach localvolunteers. Head office staff for example, can help with the development and interpretation of"flood intelligence cards", the identification of those who should be warned, the selection of warningmethods and the wording of messages. The latter is particularly important in severe events to ensurethat warnings retain credibility when responses such as evacuation are necessary.

Enthusiasm for the warning task must be generated and maintained. This can be done by instilling asense of challenge about doing the task well and by rewarding the achievement of high standards ofwarning service - for example by reporting examples of good ideas and practices in agencynewsletters and bulletins and by referring to them in conferences, exercises and debriefs. This alsohelps other people in other areas to pick up on how the job is being discharged elsewhere. Morebroadly, it encourages agencies to learn new skills and approaches and to benchmark risingstandards.

In Australia, various local initiatives have been identified and promoted, to show others how thewarning function can be discharged effectively. For example, encouragement has been given to theuse of fax machines to disseminate warnings to farmers, to the construction of pre-formattedwarning messages for radio broadcast (after fine tuning to the demands of specific events), and toproviding property holders with specific information on the gauge heights at which they will beaffected by flooding. Innovations are widely reported and used in later skills training. Withappropriate repetition, the result should be the incorporation of new methods and higher standardsof warning service delivery; in short, a general improvement in standards of practice across theagencies involved.

MONITORING AND THE DEVELOPMENT OF PERFORMANCE INDICATORS

To ensure the continuing evolution of the total system, an arrangement for performance monitoringthat encompasses all elements of the system is necessary. A focus on all elements is necessary as theeffectiveness of the total system is limited by the performance of its weakest component, and toavoid present tendencies to concentrate performance evaluations on "technical" components(prediction accuracy, reliability of data collection, etc) which are easier to measure than the

13

communication-related components that influence the quality of the protective behaviour of peopleand organisations. Routine performance monitoring of the development and operation of floodwarning systems could identify the extent to which the system is improving, and ensure thatresources are being targeted at the weaker parts of the system. Such monitoring should be anintegral part of the development of a total warning system. Reviews should occur both after anevent and between floods as system changes can occur at any time.

While the objective of a flood warning system is to maximise the proportion of avoidable damagesaved during a flood, performance indicators limited to this are not very helpful in identifying weakaspects of the system - although this is desirable to help establish priorities between alternativesystems. Identifying the difference between damage theoretically avoidable and that currentlyachieved is a first step in deciding where improvements should be introduced.

Handmer and Smith (1995) suggest an approach to assessing warning system effectiveness. Itinvolves identifying the factors limiting system performance through an audit, by breaking thesystem down into functions and components. The performance of the key components associatedwith each function is then "rated" against "desirable attributes" by those experienced with thesystem. Weaker parts of the system are thereby identified.

However, this approach may be difficult to implement and an alternative may be to combine aspectsof it with the framework of the total warning system as set out in the Guide. This has the advantagein that it reflects best practice in Australia against which it should be possible to make judgementsabout both existing and proposed systems. The five stage framework in the Guide could form thebasis for indicators of performance as suggested by Handmer and Smith (1995) and as set out inTable 3. Ratings would be provided by those familiar with the system under study. One approachmight be to rate each attribute or indicator on a 0 to 10 scale. The ratings should gradually improveas weaknesses are identified and attended to. Conversely if nothing was done, the ratings woulddrop. For this approach to be of value audits would need to be undertaken regularly, perhapsannually. Audits would be done normally on a range of areal scales - specific points (eg towns),whole river reaches (the reference areas of particular gauges), and catchments. This would allowdifferences between the various systems to be tracked, but results could be aggregated across allcatchments in a region if an overall assessment was needed.

TABLE 3 about here

CONCLUSIONS

The changes necessary for truly effective flood warning systems cannot be seen as mere events: theyrequire ongoing constant attention. Hence the need for the right institutional framework to fosterchange across the organisations involved in the warning task. Of course, this framework itself mustbe flexible and adaptive to changing circumstances and ever ready to take opportunities to improve. In Australia, a supportive institutional environment for the development of learning organisations isprovided by the FWCCs (state and territory Flood Warning Consultative Committees). The trainingarm of EMA, the Australian Emergency Management Institute, should help to provide the learningenvironment and can act as a key mediator between best practice and ordinary practice, and as aninformation source on research.

14

The Australian guidelines had their impetus in serious flooding, but the necessary institutionalenvironment was already in place in the form of the FWCCs. The key factor in developing the Guidewith a high degree of ownership by those in the flood warning business was the use of a longiterative process which was as inclusive as possible - and which itself promoted change. After theGuide was developed and endorsed by the peak flood warning body in each jurisdiction, the FWCCs,attention turned to ensuring that the process continued. In this the FWCCs were the key. Supportfrom the highest level led to a change in the mandate of the committees and to them taking on thetask of encouraging and monitoring the implementation of the Guide. The document containing theGuide is now universally and correctly seen as only part of the picture, and is itself now underrevision. Further effort is required to develop an effective process for continual monitoring andimprovement of operational practices.

How would we characterise the factors underlying what, so far at least, appears to be a reasonablyeffective process? It is difficult to argue that any one factor is more significant than others, but anumber of important elements can be identified. The process was open and transparent; the authorstried to be inclusive and to be proactive about encouraging input and comments from a widediversity of perspectives. The writing team had credibility within the Australian flood warningindustry and brought a mixture of writing skills, expertise, relevant networks (for example, onemember of the team was a member of all state FWCCs and a senior member of the national floodprediction agency), critical perspectives and independence.

However, these factors are mostly under the control of those guiding document production. Theorganisations in the flood warning process must support the activity and need to be seeking, or atleast open to, change and improvement. This is much more of a challenge than it sounds as the sortof change required may involve fundamental shifts in organisational culture and priorities - inparticular where the need is to design warning systems from the requirements of those at risk ratherthan from the perspective of the available technology or system operators. Similarly, theorganisations must be open to active participation in and accept responsibility for the process andproduct - and interested in its continuation.

The challenge now is to put the policy into normal operational practice and to make the necessarychanges self-sustaining. This is proving to be the most difficult part of the overall task of achievingchange in the total flood warning system. Several factors exacerbate the difficulties including theinevitable personnel changes and consequent loss of "champions"; and the long periods betweenfloods in what has until recently been a largely dry decade.

Obvious questions concern the time taken to prepare the Guide, and whether the process was overly"top down". The communities at risk demand improved warning services, and to that extent thewhole process of change can be seen as being driven from the bottom. Nevertheless, the responseto this demand in terms of broadening agency mandates and altering the institutional frameworkcame from the top. It is difficult to see how it could be otherwise for at least three reasons. First,there are several organisations involved with very different cultures and approaches; the challengebeing to get them to see themselves as part of a warning team - and for this institutional changesfrom the top were needed. Second, organisations such as the SES find that the bulk of their staff arededicated to rapid reactive response activities, rather than to the longer term planning issues. Changing this culture requires leadership from the top and active encouragement to preparedness-

15

enhancing activities. Third, in any given area, flooding may be rare and consequently local SESvolunteers may get little practice.

In view of the three years it took to prepare the guidelines, could they have been prepared in lesstime? There is no doubt that they could have been. But even if the writing group always gave thetask top priority, in an interactive and inclusive process the pace is inevitably dictated by others whowill see it as a lower priority than dealing with immediate problems. Rushing them may result inhalf-hearted input and therefore low commitment by these individuals. It may be that properplanning involving the relevant stakeholders is necessarily a slow process. Thoroughness is vital, andplanning must be slow and involve continuous reinforcement and repetition to effect change at thepractitioner level. Another way of viewing planning time is to see it as a permanent, unceasingcommitment. It would be possible to draw up a set of guidelines by having an intensive meeting fora few days, but this would not allow the necessary time for reflection and organisational change, andto ensure resilience in the face of constantly changing institutional and political priorities.

ACKNOWLEDGEMENTS

We express our thanks to our co-author on the "Guide": John Salter (Emergency ManagementAustralia). Many people - far too numerous to mention by name - helped with the Guide. We aregrateful to them all, and in particular to the other members of the original "working party" (DingleSmith, Australian National University; Roger Jones, Emergency Management Australia; and KenMackey, Victorian Police), our informal reference group, to all those who provided writtencomments on the various drafts, to the flood warning consultative committees, and to EmergencyManagement Australia for supporting the development of the Guide, document production andcontinuing evolution.

REFERENCES

Alexander, E.R. (1986) Approaches to planning: introducing current planning theories, concepts, and issues. Gordon and Breach Science Publishers, New York.

Branch, M.C. (1990) Planning: universal process. Praeger, New York.

BoM (Bureau of Meteorology) (1995) Flood warning service upgrade. BoM, Dept of the Environment,Sport and Territories, Canberra.

BoM and AEMI (Bureau of Meteorology and Australian Emergency Management Institute orAEMI) (1993) Guidelines for effective warning. AEMI, Mt Macedon.

Burkhart, F.N. (1991) Media, emergency warnings and citizen response. Westview Press, Boulder.

Drabek, T.E. (1986) Human systems responses to disaster: an inventory of sociological findings. Springer Verlag,New York.

16

Drabek, T.E. (1987) The professional emergency manager. Institute of Behavioral Science, University ofColorado, Boulder.

Drabek, T.E. and Hoetmer, G.J. (Eds) (1991) Emergency management: principlesa nd practice forlocal government. International City Management Association, Washington DC.

EMA (Emergency Management Australia) (1995) Flood warning: an Australian guide. EmergencyManagement Australia, Canberra. (Written by Jim Elliott, John Handmer, Chas Keys and JohnSalter.)

Foster, H.D. (1980) Disaster planning: the preservation of life and property. Springer Verlag, New York.

Gruntfest, E. (1997) Twenty years later: what we have learned since the Big Thompson flood. University ofColorado, Boulder.

Gruntfest, E. and Huber, C. (1989) Status report on flood warning systems in the United States.Environmental Management. 13 (3): 279-286

Handmer, J.W. (1997) Flood warning: issues and practice in total system design. Flood Hazard ResearchCentre, Middlesex University, London.

Handmer, J.W. and Penning-Rowsell, E.C. (1987) Lessons from Australian warning experience. InWhat have we learned since the Big Thompson flood? Proceedings of the tenth anniversary conference. (E.C.Gruntfest, ed.), pp. 189-197. Natural Hazards Research and Applications Information Center,University of Colorado: Boulder.

Handmer, J.W. and Ord, K.D. (1987) Flood warning and response. In Flood warning in Australia:policies, institutions and technology. (D.I. Smith, and J.W. Handmer, eds.), pp. 235-257. CRES, AustralianNational University, Canberra.

Handmer, J.W. and Smith, D.I. (1995) Cost-effectiveness of flood warnings. Prepared for theBureau of Meteorology, Melbourne.

Herrington, J. (1989) Planning processes: an introduction for geographers. Cambridge University Press.

Krackhardt, D. and Hanson, J.R. (1993) Informal networks: the company behind the chart. HarvardBusiness Review. July-August: 104-111.

Mileti, D.S. (1975) Natural hazard warning systems in the United States: a research assessment. Institute ofBehavioral Science, University of Colorado, Boulder, Colorado.

NDO (Natural Disasters Organisation, now Emergency Management Australia) (1989) Commonwealthcounter-disaster concepts and principles. Australian Counter Disaster Handbook, Vol 1. Canberra.

NDO (Natural Disasters Organisation, now Emergency Management Australia) (1992) Communityemergency planning guide. Australian Emergency Manual. Natural Disasters Organisation, Canberra.

17

NSW DWR (New South Wales Department of Water Resources) (1990) Nyngan April 1990: floodinvestigation. Department of Water Resources, Sydney.

Parker, D.J. and Handmer, J.W. (1997) The role of unofficial flood warning systems. Journal ofContingencies and Crisis Management. 6(1):45-60.

Parker, D.J., Fordham, M., Tunstall, S. and Ketteridge, A.M. (1995) Flood warning systems understress in the United Kingdom. Disaster Prevention and Management 4(3), 32-42

Penning-Rowsell, E.C. and Handmer, J.W. (1986) Evaluating flood warning effectiveness: the conceptual,methodological and practical research problems. CRES, Australian National University, Canberra. CRESWorking Paper 1986/6.

Salter, J. (1996) Risk management guidelines for emergency management. Emergency ManagementAustralia, Canberra.

Smith, D.I. and Handmer, J.W. (1987) Flood warning in Australia. CRES, Australian NationalUniversity, Canberra.

UK Emergency Planning Society (1998) Responding to disaster: the human aspects. (Guidance document). The Emergency Planning Society, UK. (No address provided.)

US National Weather Service (1997) Automated local flood warning systems handbook. (Weather ServiceHydrology Handbook No. 2.) Office of Hydrology, National Weather Service, Silver Spring.

Williams, H.B. (1964) Human factors in warning-and-response systems. In The threat of impendingdisaster - contributions to the psychology of stress. (G.H. Gresser, and H. Weschler, H. and Greenblatt, eds.), pp 89-114. MIT Press, Cambridge, MA.

de Vanssay, B. (1992) Les inondations du 22 Septembre 1992 dans le Vaucluse. Prepared for Ministere del'Environnement, Delegation aux Risques Majeurs. Creda-Ehess, Paris.

18

Table 1: Steps in the process of Guide development.

1. A national conference to draw lessons from the 1990 floods. Flood warnings identified asa problem area

2. 1991 national workshop on flood warnings called for "best practice" guidelines3. Time spent understanding and clarifying the task4. Ad hoc group established informally. Starts on production of a guide5. A long iterative process among a large and loose network6. Circulated draft to a few non-specialists for comment. Verdict: difficult for a non-

specialist to follow7. Redraft to make more user friendly8. Draft circulated widely. Many comments and changes9. Endorsed by all FWCCs and by EMA10. Publication of the Guide by EMA in 199511. Obtained endorsement from key Chief

Executive Officers. The guide becomes policy12. Continuing: ongoing efforts to get implementation on the

ground (to achieve permanent mindset changes)

19

Table 2: Guide structure and contents. Chapter aims and contents.

1 & 2. To set out the purpose of flood warnings and the need for a "total" or all encompassing approach - withorganisations working cooperatively with each other and the community.

The Guide then sets out each component of the system:

3. To appreciate where flood predictions are needed and why. Predictions should be based on userrequirements, linked with "informal" prediction systems, deal with uncertainty and extremeevents. They are not warnings as they simply say how high the flood will reach and when.4. Interpreting flood predictions - by adding meaning through indicating where the water will go andwho and what it will affect - turns a prediction into a warning. This demands a system forgathering and retaining detailed flood information.5. Warning messages must be designed to have meaning for the target audience, are persuasive and helpthem respond effectively. This requires knowledge of the communities and flood characteristics. A design "checklist" is included.6. Messages must be communicated through appropriate modes, so that they are both timely and likely toelicit the appropriate response. The target audience must be identified and provision made formessage confirmation.7. Flood warning systems need continuous review and improvement as circumstances change in thecatchment and stream, in technology, the communities at risk, and in the responsibleorganisations.8. Conclusions

20

Table 3: Possible framework for monitoring total flood warning system performance.

COMPONENT FACTORS AFFECTINGPERFORMANCE DURINGFLOOD

ACTIVITIES TO IMPROVEPERFORMANCE BETWEENFLOODS

RATING

PREDICTION • Understanding of predictionneeds• Early advice of floodproducing rainfall• Adequate network density• Data collection technology robust, reliable, timely• on-line data management system streamlined• Accuracy and timeliness ofriver predictions• Good communication betweenprediction agency and recipients• Good cooperation betweenofficial and unofficial groups• Good communication ofuncertainty involved

• Improve hydrological predictionmodels• Maintain data collection system• Increase network density• Improve routine catchment monitoring capability• Improve meteorological forecasting input• Refinement of prediction needs

INTERPRETATION

• Quality of available "flood intelligence"• Collection of "floodintelligence"• State of preparedness ofagency receiving prediction

• Improve understanding of floodimpacts• Floodplain mapping• Prepare flood intelligence storagesystem• Prepare system for collecting datafor next flood• Collect flood damage information

MESSAGECONSTRUCTION

• Knowledge of the physical features of the flood• Knowledge of the features ofthose at "at-risk"• Extent to which message metcriteria in "check list"

• Improve understanding of localflood features• Improve understanding of population characteristics• Work with community to improve message targeting and design

COMMUNICATION

• Use of appropriatedissemination mode(s)• Ability of those at risk to confirm warning• Appropriate mix of generaland specific warnings

• Establishment of flood information centres• provision of facilities for messageconfirmation• Work with community to improve communication

21

RESPONSE • Appropriateness of responseactions• Extent to which plans werefollowed• Feedback within system

• Preparation/improvements of flood response plan• Rehearsal of response plan• Improve inter-agency liaison

REVIEW • Undertake review at both policy-making and operational levels• Review all elements of the totalsystem after an event• Ensure review results feed intoplans

22

CIBM (Committee of Inquiry into the Bureau of Meteorology) (1977) Report: 14 December 1976. Australian Government Printing Service, Canberra.

Sinclair, P. (1990) Nyngan flood disaster: recovery report to the NSW Premier by the Premier's Special Coordinator.Premier's Office, Sydney.