Chapter 4 Research Design, Methodology & Data 62

Chapter 4 Research Design, Methodology & Data

62

4 RESEARCH DESIGN, METHODOLOGY AND DATA

4.1 Research Design Framework

This chapter presents a discussion on the research design, methodology and data. Research

design is concerned with approaches to obtaining data / knowledge, its analysis and presentation.

This is an idiographic study which is concerned with the study of a social phenomenon in its

social setting i.e. learning within information systems development in Botswana and hence

qualitative research methods and not quantitative methods which are commonly used in the

natural sciences for the study of natural phenomena have been employed. According to

Myers(1999):

‘Qualitative research involves the use of qualitative data, such as interviews, documents, and participant

observation data, to understand and explain social phenomena… In Information Systems, there has been a general

shift in IS research away from technological to managerial and organisational issues, hence an increasing interest

in the application of qualitative research methods’. (Myers (1999, p.1)

In this research study I used a plurality of qualitative research methods as part of the research

design. According to Myers (1999), the most commonly used qualitative research methods are

action research, case study research and ethnography. But lately, in activity theory based studies,

DWR has been adopted as the research method. I use a single case project to obtain initial

understanding of the nature and complexity of the research context (Myers, 1999). This

particular project was selected for this study because findings from its post implementation

review indicated that there was slow systems uptake which was consistent with my concerns

about learning in current practice. The use of a single case, together with interview data from an

interview with a representative of the GIT, was deemed sufficient to delineate the current ISD

activity and to provide the initial data used to mirror the practice. This assisted in setting the

scene for practitioners to identify problems and to begin to question aspects of the current ISD

practice. The case analysis also provided me with the basis for selection of participants for the

subsequent change labs. Because case study research and ethnography only allow for observation

by the researcher with no intervention, I had to consider the use of action research or the activity

theoretical based DWR which are interventionist approaches to accommodate the intervention

aspect of the research.


63

Between the two, I opted to use DWR because it is based on CHAT concepts. Action research is

a more generic method that can be used with any framework. Furthermore, in its attempt to

ensure scientific rigour, action research requires that all the five steps have to be completed and

reported on as part of the research study (Grant and Ngwenyana, 2003; Baskerville and Wood-

Harper, (1996,1998), whereas DWR provides more flexibility, even in terms of the starting point

and end points of the research. It’s important to note here that even though we talk about steps in

DWR / the expansive learning cycle, these are ideal typical steps and do not necessarily have to

be carried out in that order (Virkkunen and Kuutti, 2000; Toiviainen, 2007; Engeström and

Sannino, 2010). In fact, according to Engeström and Sannino (2010),

‘… it is not a universal formula of phases or stages. In fact, one probably never finds a concrete collective

learning process which would cleanly follow the ideal-typical model. The model is a heuristic conceptual

device derived from the logic of ascending from the abstract to the concrete. Every time one examines or

facilitates a potentially expansive learning process with the help of the model, one tests, criticizes and

hopefully enriches the theoretical ideas of the model.’ (Engeström and Sannino, 2010, p.7)

Engeström and Sannino (2010, p. 3) also point to a number of studies that have adopted the use

of DWR / Expansive Learning because it ‘… has been found particularly useful in analyses of

learning in non-traditional, hybrid and multi-organisational settings’, which are similar to this

research study.

DWR is a specific intervention methodology developed by Engeström (1987) that allows for the

application of CHAT concepts as discussed in Chapter 2, to work development. DWR follows

the epistemic actions as described in Chapter 3 for expansive learning and as such allows for

carrying out learning activity in collaboration between a researcher interventionist and

practitioners.

In DWR the role of the researcher as the interventionist is to facilitate a process of undertaking

the epistemic actions and create an environment where practitioners begin to analyse the current

activity based on its historical development and to think about an expansive transformation of the

activity system.


64

The research setting for DWR, which provides practitioners with the tools for taking

collaboratively expansive learning actions, is based on Vygotsky’s idea of developmental

experiment which is based on the method of dual stimulation. The purpose of these

developmental experiments by Vygotsky was to show that potential capabilities and emerging

new psychological formations in a child could be identified by analysing the formation of new

meanings or new generalisations.

The first stimulus in the DWR process as depicted in Figure 10 is provided by the researcher to

the practitioners in the form of data about problematic aspects of their daily activities and

disturbances in them in order to trigger an identification of the need for change. This could be in

the form of videotaped data or any other similar data. The general model of an activity system is

used as an intellectual tool for modelling both the systemic cause of the identified problems and

a new form of activity during the DWR process. This acts as a second stimulus. The third type of

tools / artefacts that are used as instruments of expansive learning actions are analytical concepts

and various representational means for analysing the data in the mirror and for constructing

alternative solutions for specific parts of the activity.

��

��

��

��

�� !��"��#

��$��

��%� ��&��

��&��

��

$��'�(��

)��*��+

,�"�&�)� ��

-��.��

/ ��#0 ��

��

�� '�� )� ��

Figure 10: Developmental Work Research Schematic / Design (adapted from Engeström, 1999, p. 7)


65

About the interventionist and experimental nature of the DWR approach, Engeström (1999)

states:

‘This type of design requires a bold experimental attitude rather than an attitude of a casual observer and

facilitator. Bringing about and traversing the collective zone of proximal development is experimentation

with activity systems. When practitioners face a mirror depicting their own disturbances, they often

experience them as personal failures or even crises. Powerful and unpredictable cognitive, emotional and

social dissonances are triggered.’ (Engeström, 1999, p. 7)

The phases of DWR / expansive learning were presented and discussed in Chapter 3 (Section

3.3.5). The epistemic learning actions as identified there were used for this research study

starting with obtaining initial field work data from the case project and using that as the first

stimulus. After delineating the current activity system I moved on to the next DWR intervention

phase and carried out the historical analysis and actual-empirical analyses of the ISD work

activity as a historically developed local system. On the basis of this analysis a new ISD activity

system was modelled tackling current challenges and contradictions. This was later examined in

a change laboratory session.

The Change Laboratory (CL) method, which is commonly used with DWR interventions, is a

method that was developed in the mid-1990s by researchers from Helsinki University. The CL

provides an environment for the various participants to think through and experiment with new

ways of working. It is particularly useful in supporting practitioners in the theoretical-genetic

reflection on joint activity as well as on conceptualising new practice through interactions and

discussions. The CL provides an opportunity for participants to engage in a collaborative

learning activity by carrying out systematically expansive learning actions of questioning current

ways of thinking and working, analysing and modelling the activity system, and conducting

thought experiments concerning possible changes to their work activity. During CL sessions,

participants have been observed to move from singular individual positions to the collective

agent i.e. from ‘I’ to ‘We’, which therefore requires the formation of new and shared tools, rules

and division of labour. In the CL the interventionist facilitates a process whereby participants

move between the past, the present and the future activity (Engeström et al., 1996; Engeström,

2001; Pihlaja, 2005; Hill et al., 2007).


66

In the CL method, implementation of the designed new solution, which is done in the form of

pilot experiments, usually starts while the CL sessions are still running. The implementation

typically leads to a richer and more articulated concept as solutions addressing contradictions

arising from the implementation are used to enrich the concept. The change laboratory process

typically comprises of five (5) to ten (10) sessions of two (2) to three (3) hours (i.e. 10 to 30 hour

sessions) and a varying number of follow-up sessions. Participants comprise of practitioners and

managers and where possible, users, customers or patients are invited to the sessions in order to

create the necessary tensions as their cases, problems are being looked at. Data collected from

the activity setting relating to critical incidents and problems in the current work practice are

brought to the CL sessions and play a major role in stimulating discussion, analysis and

collaborative design efforts amongst the participants. Because the CL sessions themselves are

videotaped for analysis it allows for the collection of rich longitudinal data on the discussions,

actions and interactions involved in the collaborative learning activity (Engeström et al., (1996);

Engeström, 2001; Pihlaja, 2005; Hill et al., (2007)).

The change lab method was used for this research study, but instead of short 2 to 3 hour sessions,

I only had two sessions of seven hours and five hours respectively. I restricted myself to two

long sessions because I had already anticipated that it would be difficult to get representatives

from government and the private sector if I had more sessions.

Most DWR (and Expansive learning) studies that have been carried out tend to use ethnographic

research for the initial field work and data collection (Hasu 2001; Toiviainen, 2003; Pihlaja,

2005; Kerosuo, 2006). Ethnographic research, which draws its roots from the discipline of social

and cultural anthropology is characterised by the long period of time a researcher needs to spend

in the field. The ethnographic researcher is totally immersed in the research environment as they

attempt to gain a deep understanding of what people are doing as well as the environment around

them. Though in this particular study I did not conduct an in-depth ethnography in terms of

trying to ‘… follow the people, follow the thing, follow the metaphor, follow the plot, story or

allegory, follow the life or biography, and follow the conflict (Kerosuo 2006, p. 93)’ on a day to

day or sustained basis, as such, I was around the project long enough ((i.e. from 2004 to 2011) to


67

observe what was going on. Even after the project had been completed, I continued to visit the

case site to interact with the users and find out how the system was doing.

The research framework for this study was depicted at Figure 1 (in Chapter 1) and it provides a

summary of the research approach. In the research framework the literature review, case project,

interviews, observations, archived reports and change labs were used for analysis of current ISD

practice. The literature review covers literature on current ISD practice and moves to a

theoretical analysis of situated learning theories since the research is concerned with situated

learning in the context of ISD. In 2006, I decided to carry out this research study and sought

permission from the Government of Botswana, and more specifically the then Ministry of

Communications, Science and Technology (MCST) to use ISD projects that I was engaged in as

a consultant for my research. At the same time I also sought permission from the respective

government departments who ‘owned’ the projects as well as the two private companies that had

been contracted to carry out development on the two projects I was engaged in at the time and

had earmarked for the research. The necessary permissions from MCST and other relevant

parties were granted during the first quarter of 2007.

The literature review and more specifically the case analyses provided the basis for the

collaborative redesign effort. User representatives in the analysis and redesign change labs were

drawn mainly from the two projects that I was engaged in at the time and for which permission

had been granted for research. This research study only reports on the analysis and redesign

processes and the learning that has resulted from that. The testing, implementation and diffusion

will be reported on as part of future research.

4.2 Research Questions and the Unit of Analysis This research study is concerned with four research questions.


68

Table 4 provides a summary of the research questions, data, unit of analysis and the activity

theory concepts used for analysis.

1. What constitutes Botswana’s ISD practice?

This first question seeks to understand the context of the study using the activity system

concept. I identify and describe the elements of the Botswana ISD practice in terms of

subjects, object, tools, rules, community and division of labour. I further analysed the

contradictions within and between elements of the activity system as part of understanding

the context of the study. In addition to the use of the activity system and contradiction

concepts, I also use the concept of historicity to carry out a historical analysis of the activity

based on archival data as well as interviews with a key government representative. This

constitutes the initial fieldwork of the study.

2. What are the Users and developers learning and is the learning effective?

In terms of the context of the study, there are two groups of subjects identified i.e. the Users

(i.e. all those that represent government and therefore include functional users, user

management, user IS practitioners and representatives of GIT) and the developers (i.e. those

that represent the private sector). The study seeks to establish what these two groups of

subjects are learning during current practice and establish whether the learning is effective.

Effective learning, as argued in chapter 1, will facilitate higher levels of (user) system uptake

and meaningful work practice improvement. The meaningful work improvement is

something that both the users and developers contribute towards as they engage in

development of a new IT artefact. The analysis of learning and its effectiveness was done

retrospectively, that is post-implementation and it was carried out using a model that

combines Leontiev’s concept of the hierarchal nature of activities i.e. the distinction between

activity, actions and operations and a heuristic model based on the distinction between formal

and informal learning as suggested by some learning theorists (e.g. Rogers, 2003; Malcolm et

al., 2003). Learning tasks in the case project are identified and classified as either ‘conscious’


69

or ‘unconscious’. The conclusion as to whether the learning was effective, is therefore, based

on the balance found i.e. was there a good mixture of conscious and unconscious learning in

the learning tasks? or was there more conscious or unconscious learning found?. The main

data used here is from the case project.

3. How can current ISD practice be improved in order to facilitate effective learning?

Once an understanding of current ISD practice and learning was obtained the study moved on

to trying to come up with a solution to the current learning problem. This was done through a

collaborative redesign effort with users and developers in two CL sessions where users and

developers were represented.

4. What do Users and developers learn when collaborating in the analysis, review and redesign

of ISD practice?

The concept of expansion is used in the analysis of the learning that takes place during the

analysis, review redesign of new ISD practice. I look at the interaction of the different social

actors with the common object in terms of how they perceive what can be done to improve

current practice. I study the multiple voices coming from the different representatives during

the change labs / CL sessions and try to understand the ease or difficulty with which the

social actors are able to create a shared understanding of what is going on in the process

(Hasu, 2001). I also analysed the nature of the expansion that takes place using the four

dimensions of expansion of an object as described by Hasu (2001) and Engeström and

Sannino (2010).


70

Table 4: Research Questions Summary

Research Question Data Unit of Analysis Analytical Concepts

Used

What constitutes

Botswana’s ISD practice?

Interviews,

video data from the 1st

change lab session, and

archive reports

Botswana ISD network

of activity systems

Activity system context

analysis;

Multivoicedness;

Historicity;

Contradictions

What are the users &

developers learning and is

the learning effective?

Observations from the

case project;

Interviews with Users and

Developers from the case

project;

Analysis of learning

actions from the ISD

activity system

Activity system

hierarchy; Heuristic

model based on two types

of learning i.e. conscious

and unconscious learning;

How can current practice

be improved in order to

facilitate effective

learning?

Video data from change

lab sessions

Network of activity

systems i.e. Users &

developers activity

interacting systems

Expansive learning;

Reflection-on-practice.

What do users and IS

professionals learn when

collaborating in the

review and redesign of

ISD practice?

Video data from change

lab sessions

Interacting activity

system of Users and

developers

Expansive learning;

Object expansion in

various dimensions.

4.3 Data Collection I used a wide range of data sources for this study, which is consistent with the adopted DWR

research method. I used interviews, observations, archives, and CL sessions for data collection.

The data summary is provided in Table 5.

Interviews

Other than interviews conducted with users and developers as part of the post implementation

review of the case project as well as the second project, interviews were conducted with

representatives of the government department responsible for coordination and policy direction

of all government IS / IT matters. From now onwards I shall refer to the government IT

department as GIT and the representative from that department as the GITREP.


71

One interview, specific to ISD, was conducted with a GITREP sometime in 2007 soon after I had

been granted permission to proceed with the research. This was intended to set the scene or get a

confirmation from government on the current ISD process. The interview was open ended and

based on a guideline as opposed to specific questions. I asked questions as they arose from

discussions with the GITREP but with the ultimate aim of getting an understanding of the

process from the government perspective.

I also used interview data from interviews conducted for the post implementation review of the

case project. The PIR data used was from interviews of fourteen (14) users that were interviewed

in December 2007 and due to other work commitments, the Company X (COX) representatives

were only interviewed in March 2008 and even then it was only possible to interview two (2) out

of the five (5) that were involved in this project. The two that were not interviewed were difficult

to trace because one of them left the company immediately after the development was complete

and the other one was engaged as a free lance consultant and was not a permanent employee of

the company. The final member of the development team, who also was the Lead Developer /

Project Manager was only interviewed after the first CL session where he was also a participant.

Though the interview was delayed it worked out well for the research because it gave me an

opportunity to revisit, with him, some of the points he made during the first CL session.

For both users and developer the interviews were based on open ended questions (i.e. refer to

Appendix A, B and C) and though the questions were in English, for most of the users they were

asked in the local language (i.e. Setswana) for ease of their understanding. The responses by the

users were given using a mix of both English and Setswana, depending on the language the user

felt most comfortable conversing in. I translated and recorded the responses in English.

Observations

Since I had been engaged with the project since 2004, I have also used some of my notes and

observations that I made during the course of the project. These notes and observations are not

extensive and systematic in any way since at that time I had not yet received the necessary

permission to proceed with the research. Though not extensive, they have been found useful in

providing insight into current ISD practice.


72

Archives / Secondary Data

Archive data has been used as a secondary data source. The archive data has mainly been used

for the initial case analysis, especially as it related to historicity. I used archived research reports,

minutes from project meetings and other relevant reports such as the Government of Botswana

National Development Plan reports spanning the period 1966 when Botswana attained its

independence to now.

Change Lab Sessions

Two change labs were conducted as part of the research study, with participants drawn from the

GIT, functional users from the two projects approved for the study and all seven companies

known to offer system development services and operating in the country. The first change lab,

which ran for seven hours, held in November 2007 was attended by 22 participants from

government and the private sector. The major objective of this first session was to introduce the

research and to carry out the initial analysis of what constitutes current practice. This initial

change lab was conducted differently to how most DWR lab sessions are conducted in that I

opted to have different social actors make PowerPoint presentations on 1) the historical aspect of

current ISD practice, 2) how they were currently carrying out ISD projects 3) current

contradictions and 4) how current practice may be improved. The presentations were then

followed up with a discussion by participants on current contradictions and solutions to the

problem. The reason for this approach was that I wanted to garner support for my study from the

participants and I also wanted them to feel from the onset that they were involved in not only the

analysis of practice, but also looking for solutions. In terms of the change lab programme, I had

asked participants to focus on the following discussion points:

• Specific Botswana ISD project experience

• ISD methodology, techniques used including justification for choice of methodology

• Suitability of ISD methodology to specific projects

• Meanings that users in particular assign to the ISD methodologies used

• What learning took place on the projects by the different social actors i.e. users, and

developers

• Suitability of chosen methodology to system uptake and learning


73

• ISD practice challenges

• Suggestions for improvement

Prior to presentations and discussions by the developer firms on the points listed above, the

GITREP set the scene by taking participants through the historical development of the current

practice as well as outlining what in their view were the current challenges government was

experiencing in ISD projects. The user representatives also then shared their experiences and

what they perceived to be challenges with the current practice.

Before these presentations by users and developers I had given a presentation that explained the

research problem & objectives as well as covering CHAT concepts that would be used for the

research. The presentations also included a representation, based on the interview with the

GITREP, as well as my own observations, of the current ISD activity system using Engeström’s

activity system triangle.

The second change lab, which ran for about five hours, was conducted in April 2011 and during

this change lab I presented a model, based on suggestions from the first change lab as well as my

own input on what the improved practice should look like. After presenting the new practice

model I gave participants the following guideline questions for discussion: 1) Would this (i.e. the

new model) suffice and will it address the current learning challenges; 2) How can we make it

work in practice? Do we need to have a ‘Learning Contract’ in addition to the Memorandum of

Agreement between clients (i.e. users) and suppliers (developers)?; 3) How and when can we test

and implement this model? What needs to be done in order for us to implement and test this new

model (e.g. does the current government IT framework need to be amended?).

This second change lab did not have the presence of a GITREP whose participation was deemed

important for the adoption and diffusion of the new model. However for purposes of the practice

redesign, their suggestions for improvement to practice had already been provided at the first

change lab and therefore incorporated in the new model. The adoption and diffusion into practice

will be pursued as a separate study and will be reported on outside of this research.

Data from both change labs was videotaped for analysis. The use of video recording for these

change labs was convenient for me because I then did not need to take down any notes and it

allowed me to fully participate in the discussions. The recorder captured all of the proceedings in


74

full and even though the video recording was done by one of the Activity Theory Interest Group

(ATIG) members, I took responsibility for transcribing all the data because of my familiarity

with the data and also because the transcription also included some form of interpretation and

analysis.

Table 5: Research Data Summary

Type of Data Number

1. Interviews

• Interviews with GIT on Technology related matters 3

• Interviews with GITREP on ISD 1

• Interview with Functional Users from PEX project 14

• Follow-Up interviews with Functional Users of PEX project 5

• Interview with PEX Management 4

• Interviews with Developers of PEX project 2

• Follow-Up Interview with Developers of PEX Project 1

Total Interview Data 30

2. Observations

• Prototyping Sessions 3

• User Acceptance Testing sessions 2

• Training sessions 2

• System Piloting / Hand Holding 2

Total Observations 9

�� Archive / Secondary Data�

• PEX Project Reports 15

• PEX Project Minutes 20

• PEX Training Evaluation Forms 14

• Selective ISD Project 2 Documentation 6

• National Development Plans 5

• GIT Document Framework Library 1

Total 61

4. Change Lab Sessions

• Change Lab session November 2007 1 seven hour session

• Change Lab session April 2011 1 five hour session

Total 12 hours of change lab sessions (~ 6 CL’ s of

two hours each)


75

4.4 Researcher Role

As stated previously, as the researcher I was also one of the consultants involved in the case

project as an analyst initially (i.e. we developed the user specification) and later on providing

project management support during the design and development phases. The project

management role necessarily placed me on the side of the IS users and thus an active participant

involved, amongst other things, in ensuring the quality of the developer firms products and

services. Furthermore, I have a long standing relationship with some of the participants from the

GIT, as a previous employee of that department. In recognising and acknowledging these

relationships upfront, I want to ensure that the validity of the findings is not compromised in any

way. But, as observed by Dwyer and Buckle (2009), in terms of the qualitative nature of the

research, the ‘situatedness’ of the research means that the researcher (as subject) cannot divorce

herself from the research (object). Being an insider provided me with easier access to the data as

well as the potential candidates for interviews and change lab sessions. But once the data had

been collected I tried to distance myself from the insider role and assumed the outsider role of

analysing the data. This insider-outsider role is supported by the adopted DWR research

methodology.

4.5 Conclusion

In this chapter I have provided a discussion on the DWR research methodology adopted for this

research. DWR was selected because it provides a detailed framework based on CHAT concepts

which ties in with the objectives of this research. The chapter also provided a detailed discussion

of the data that was used to address each of the research questions. The initial field data used as a

‘mirror’ to analyse and reflect on current ISD practice was mainly based on a single case project

which I was engaged in at the time that permission was granted for me to carry out this research.

In fact at the time permission was granted, the post-implementation review had just been

concluded with one of the major findings being that of slow systems uptake, which I had

identified as one of the manifestations of ineffective learning in ISD practice. This, therefore,

made this particular project an ideal case for my research. My role as researcher, practitioner and

interventionist, which is acceptable for the chosen DWR methodology, has also been explained

in this chapter.

Chapter 5 Current Botswana ISD Practice

76

5 CURRENT BOTSWANA ISD PRACTICE

5.1 Introduction

In this research study I set out to involve IS practitioners from government and industry, as well

as users, in the collaborative review and redesign of current ISD practice and therefore as a

starting point it is important to have a detailed understanding of how ISD is currently being

carried out in Botswana. The description of current practice is provided in this chapter starting

off with identification and description of all the actions/tasks carried out in current practice. The

description provided here is based on interview data from interviews with the GITREP,

functional users and developers on the selected case project, as well as on secondary data i.e. the

GIT Framework library, the case project reports, and reports from the second project that I was

engaged in at the time.

5.2 The Current Botswana ISD Practice Model

As stated earlier, the formal description of how ISD projects are currently being carried out in

government was provided by the GITREP in an interview that was conducted in May 2007

which was around the time that I had received approval from government to proceed with my

research. According to GITREP, ISD projects are carried out following process steps which are

aligned to the Software Development Life Cycle (SDLC) i.e. Analysis-Design-Development-

Implementation, with formal sign-offs expected at the end of each stage (refer to Figure 11). The

process was in accordance with the GIT framework which was developed in 1999, to support the

decentralisation of IT services from the GIT to other government departments. The framework

was intended to provide guidance in all important areas of IT within Government including IT

Management & Planning, IT Service Management, IT Procurement, and IT Security. It also

defines procedures and standards that are to be adhered to by Third Party suppliers of IT

products and services to Government. For example standards have been included in the

framework for production of a Project Initiation Document (PID), Statement of User

Requirements (SOUR), Requirements Catalogue, Invitation To Tender (ITT), Acceptance Test

Specification (ATS), Procedure Definition.


77

Figure 11: The Current Botswana ISD Practice Model

The trigger of any ISD initiative, according to the GITREP is the indication of intent by a User

Department and about this the GITREP stated:

‘For commitment and sponsorship we want users / clients to initiate the process, indicate

requirements, reasons and benefits (for embarking on an information systems

development project). But (they) go into motions of computerisation not because they

understand the responsibility but because of appreciation of ICT as enabler. They believe

that IT can help solve problems’

This indication of intent is done in the form of a formal letter either to the GIT Department or

their representative in the specific User Department. Depending on the nature and size of the

project, this is followed by production of a budget for conducting detailed analysis of the

requirements and production of the SOUR. Currently, this is an activity which is outsourced and

the methodology and approach of developing the SOUR is determined by the analyst firm. But

the minimum standards for a SOUR document have been articulated in the GIT Framework. The

design and development of the software solution is also outsourced and here again the

methodology is determined by the supplier. In terms of project management, government has


78

adopted PRINCE2 (Projects in a Controlled Environment 2) as the project management

methodology and training of all IT officers across government was to be initiated.

I will now unpack this high-level description by the GITREP using the selected case project,

which is fully representative of how ISD projects are currently being carried out in Botswana,

since suppliers are all expected to adhere to the GIT framework.

5.3 The Case Project

The selected case project was, to a large extent, carried out in the manner described by the

GITREP. The case project was for a public entity X, which I will now refer to as the PEX.

Before going into the details of the project itself, I will provide a brief background on the PEX

organisation. This is then followed by a description based on the ISD process stages as outlined

by the GITREP i.e. the Requirements or SOUR stage, Design and Development stage, as well as

Support. The description of these stages is presented following the methodology as adopted by

the third party suppliers.

In addition to a description of the actions during the design and development stage, I also

provide a summary of the outcome of these actions i.e. the product or IT artefact that was

developed because even though the quality of the product is not being analysed as part of this

research study, it is indeed important to know about the end-product.

On the PEX project, though not standard practice, a post implementation review was carried out

and later on in this chapter I have reported on the findings. The findings of this PIR are of

particular interest to this study because they include an assessment of learning on the project.

5.3.1 The PEX Organisational Background

The core business for PEX, which is a unit within a much larger government research

department, is the multiplication of seed for the arable farming community. The PEX operates

from offices that are located outside of the capital city, Gaborone where they have a large land

area for growing and multiplying seed.

The PEX mandate of seed multiplication is achieved through the contracting of external farmers

(or seed growers) to grow the seed which is then supplied to the farmers by PEX, or through

growing the seed internally, on PEX farms. During the planting-growing-harvesting period, PEX


79

monitors the growth and quality of the seed produced through regular inspections of the

contracted Seed Growers farms or their own farms. Once the seed has been harvested it is

processed and packaged ready for sale to the wider farming community.

The PEX executes its mandate through five business units, namely: Inspection (Certified Seed),

Seed Laboratory, Basic Seed Production, Processing and Despatch (refer to Figure 12 for the

PEX functional structure). The system that the PEX required was to support all these five

business functions.

Figure 12: PEX Functional Structure

The Inspection (Certified Seed) Sub-Unit deals with the external seed growers in terms of

contracting them to grow seed as well as monitoring the seed growth through the planting-

growing-harvesting cycle. The Basic Seed function, on the other hand, is responsible for

monitoring growth of seed that is grown internally using breeder seed obtained from an

independent research arm of the same department that the PEX belongs to. Once harvested this

basic seed is distributed to the external seed growers for multiplication. Seed quality is important

to the PEX and this is provided by the Seed Laboratory as a support service to the Certified Seed,

Basic Seed Production and the Warehouse (i.e. Seed Processing & Despatch) Sub-Units. The

same services are also availed to external users from the private sector. This specialised service

involves carrying out various tests, based on local and international testing standards, to


80

determine the quality of the seed produced. The most common tests carried out are: Purity Test,

Moisture Test and Germination Tests. The Processing Sub-Unit is responsible for cleaning,

grading, treating and packaging both basic and certified seed. At the time the project was carried

out, the Unit had three warehouses, one open shed and one cold room where the basic and

certified seed are stored for future sales and dispatch. In order to maintain the quality of the seed

the storage facilities are kept clean at all times and in cases where existence of weevils has been

identified, the facilities are fumigated and sprayed. The Despatch Sub-Unit is responsible for

seed sales and despatch to all types of client i.e. from government or the private sector. The

breakdown of PEX staff by business unit is as shown in Table 6.

Table 6: PEX Staff Profile

Business Unit Professionals Artisans Technicians Farm

Workers

Others

(e.g. Stores)

Total

Inspection 1 2 3

Seed Lab 2 2 4

Basic Seed Production 1 1 8 10

Processing 2 2

Warehouse/Despatch 1 2 3

Total 2 3 7 8 2 22

The table shows that most of the PEX staff comprises of technicians and farm workers. This is

the pool of resources from which the functional users of the PEX project were drawn from. At

the very top of the PEX structure is the Head of the Organisation and the Deputy who the PEX

had direct reporting lines to. It is at this level of the organisation that the need for the PEX

system was conceptualised. The analysis of how ISD projects are conceptualised will later show

that there is a contradiction between the level at which decisions are made and participation in

the project.


81

From the training needs analysis that was carried out for the new PEX system, it was concluded

that users did not have sufficient knowledge of computers, and therefore as part of the delivery

of the PEX system, basic computer training was to be provided first, together with basic training

on desktop applications i.e. MS Word and MS Excel.

Prior to this project, the PEX or indeed the department within which the PEX was located had

not been engaged in a large scale ISD project of this nature, especially with external service

providers involved. This was therefore a new experience to the staff and the organisation. From

the onset it was a challenge to make sure that the right people at the right levels were involved in

the project. The culture was such that IT projects were for IT people and so both management

and users did not think that their input was crucial.

5.3.2 PEX Project Conceptualisation

Over a number of years PEX was experiencing problems and limitations with data collection,

management as well as forecasting of seed demands for upcoming seasons. The PEX was using a

standalone DOS based database system that was developed in-house using Dataease (i.e. a 4GL

database management system). The system had been in use since 1996 and it only provided for

basic information input and output (e.g. relating to daily seed processing, receipts from farmers /

seed growers, (basic) seed despatch to farmers / seed growers, etc), with access limited to one or

two individuals on a standalone environment / computer. In fact what used to happen was that

various forms were submitted to a central location (IT Office) where data was captured on the

system followed by the production of standard reports. This movement of forms was blamed for

the sometimes incomplete information as forms were lost and did not reach their destination. The

limitations with the current system, together with advances in information and communications

technology were cited as some of the reasons for wanting to embark on this project with the

major aim being to develop and implement a more comprehensive system that would improve

the PEX practice. According to the PEX IT Manager, the decision to develop a new system to

improve the PEX work practices was taken by Management, and more specifically the previous

Director of the organisation. At the time that I became involved with the project a new Director

was in post and from the interview I held with her in May 2004, she expressed concerns about

their inability to forecast seed demands and thus ensure that they always have the required seed

quantities to sustain the nation for a period of time, including during drought periods. This, she


82

said, was one of the major reasons a new PEX system was required. These concerns were shared

by her immediate deputy, who was a direct supervisor of the PEX unit, who commented that:

‘We always need to have enough seeds for two seasons ... and so we need to be able to

forecast demand. We also have to distribute seeds for free to farmers during a drought,

but we do not know what the national seed requirement is. ’

The decision to develop a new system was, therefore, a management decision with little or no

involvement of staff. It took ten (10) years from conceptualisation in 1997 to the PEX system

implementation which was only completed in 2007. The SOUR was completed in 2004, and the

design and development only commenced two years later in 2006. This time lag presented

challenges in that there were significant staff movements during that period, including the

departure of the Director who had conceptualised the project.

The PEX project was made up of two components i.e. the IS component and the IT or

technology component which was concerned with the supply, delivery, installation and

commissioning of hardware artefacts. The two project components were carried out in parallel

with the hardware delivery timelines such that all equipment was in place by the time the training

and testing of the PEX solution begun. The focus of this study is on the IS side of the project

delivery.

5.3.3 Project Management Process

The project management process followed guidelines as provided for in the GIT framework, with

a Project Initiation Document (PID) produced as the initial output outlining how the project was

to be carried out, managed and controlled. The PID provided for sign-offs at each stage of the

process and more specifically for each of the project deliverables. The sign-offs as specified

were done religiously on the PEX projects, however, there was no assessment of the

understanding of all those involved at each stage of the project. It was assumed, as was practice

that everybody was on board. It only struck me that not everybody was on board when one of the

users during testing of the system said:

‘Aha ... OK ... this is what we wanted to do all along ...now I understand’

This was a telling statement, which contributed to my decision to study learning within ISD and

it also influenced my decision to use this particular project as the case project for this study.


83

The project structure comprised of a Project Implementation Committee, a Project Manager, and

the Functional User Team supported by the PEX IT staff. The structure, as depicted at Figure 13,

shows at a high level the social actors that were involved in this project. This is very typical of

ISD projects in Botswana. The social actors represented here participated in the two change lab

sessions to review and redesign practice.

Figure 13: PEX Project Structure

The (PIC) was responsible for, amongst other things, project coordination, and deliverable

quality reviews. Membership of the PIC comprised of the head of the PEX, functional users (i.e.

from the five sub-unit of Inspection (Certified Seed), Seed Laboratory, Basic Seed Production,

Processing and Despatch), representatives of the analyst firm, the developer firm, and

representation from the IT function of the Ministry to which the PEX belonged. Though the PEX

Management was not represented in the PIC, there was provision for them to receive regular

briefings from the Project Manager. This, however, did not happen as regularly as had been

envisaged due to their unavailability. The Project Manager was the IT Manager for the PEX,

supported by the analyst firm that had developed the SOUR. The analyst project management

services were provided on a part-time basis and were only for the development stage of the

project – the support phase was not included. The profile of the functional users is provided in

Table 7 below. The profile shows that most of the functional users had diplomas and certificates


84

most of which were acquired thirteen to thirty eight years ago. Selection of functional users was

done by the Head of the PEX and where possible, the same team that was involved during the

SOUR phase of the project was retained for the design, development and implementation. But,

throughout the project it was a challenge retaining the same people.

Table 7: Profile of Functional Users involved in the PEX Project

Level of Qualification No of Users Year Obtained

Masters 1 2002

Degree 1 1992

Diploma 5 1988, 1990, 1998, 2006

Certificate 5 1973, 1981, 1988, 1990

None 1 N/A

In terms of project control, the PIC met fortnightly for progress monitoring and control, as well

as at the end of each stage for deliverable quality reviews. There was also provision for the PIC

to meet on an ad-hoc basis in case of exceptions that needed to be addressed urgently.

5.3.4 PEX system requirements

The PEX system requirements elicitation and documentation was done by an external supplier –

the analyst firm, of which I was the lead analyst. The choice of methodology for development of

the SOUR was left to the supplier and in this case we used a combination of the Critical Success

Factor (CSF) method by John F. Rockart (1979), and structured analysis techniques through the

use of data flow diagrams. The final SOUR document comprised of a description of the current

system (i.e. using DFDs) as well as a simple non-technical description of the requirements.

Requirements catalogues as well as a data dictionary were also produced in line with the

requirements of the GIT Framework. The SOUR requirements were obtained from the PEX

Management and functional users through change labs and interviews. Full day change labs were

conducted for each of the PEX functions / sub-units where requirements elicitation was

facilitated by the analyst firm.

The details of the PEX system requirements have been documented in the SOUR (2004)

document following the GIT standard for such documentation. Requirements were specified for


85

each one of the PEX functions. Some of the key requirements included the ability to capture all

details of a seed grower based on their application and also the ability to automatically vet the

applications based on predefined criteria. The system was to allow flexibility in setting up the

vetting criteria for such applications. Once the seed grower had been contracted through the

system, their performance was also to be monitored through the system. This was to be carried

out through the maintenance of inspection report findings captured at each stage of the growing

process. In terms of the inspections functionality, a scheduler was required to assist the

inspectors in scheduling inspections including having some sort of prompts or alerts for

inspections that are due. In order to address the Seed Lab requirements, the PEX system was to

provide the ability to receive samples through capturing sample details into the system and

generating a unique laboratory number for each sample. Once the testing and analysis were

complete, the results would be captured into the system and the relevant reports generated

including the test certificate. This seed lab functionality was meant to support testing of samples

for both internal and external seed growers.

In terms of Seed Processing, the system was to provide functionality for receipt of seed from

various sources including interfacing with the existing weighbridge system as well as production

of seed pocket labels for packaging the processed seed. The warehouse functionality as provided

for in the SOUR included the ability to maintain warehouse details in terms of their location and

size / capacity. It also included the ability to receive seeds into the warehouse, allocating it for

storage in specific areas in the warehouse, issuing of seed four sale and monitoring of stock

levels. In terms of the key non-functional requirements, the new system was to be modular with a

Graphical User Interface (GUI) (i.e. from DOS to windows based) operating in a secure multi-

user environment. Finally a number of standard reports were specified for generation by the new

system (e.g. Seed Growers monitoring report, Seed Inspection Reports, Seed Testing Certificate,

Seed Pocket Labels, Seed Processing Report etc.).

The new system would be made available to all PEX functional users with data captured at

source. As a result there would no longer be any movement of forms for centralised data capture

and report processing. Furthermore access was to be provided to management for viewing /

printing relevant reports.


86

The requirements as specified for the new system were a significant improvement to the existing

system and therefore the system was expected to address most of the user concerns and yield

significant benefits (PEX SOUR, 2004). The anticipated benefits would be the ability to monitor

the performance of Seed Growers, since all information relating to their contracts and findings

from inspections would be in the new system. The maintenance of inspection reports was

expected to help improve inspection management including ensuring that scheduled inspections

were not missed since the system would provide alerts. The benefit of having a Lab Test module

was viewed as an improvement in seed analysis as it ensured that international standards for

testing of seed were adhered to since the functionality would be built into the new system. The

enforcement of standards for say the picking sequence (i.e. in this case FIFO) when seed was

being issued, would be a major improvement to current practice where the picking sequence was

not consistent resulting in old seed staying in the warehouse for long periods at the risk of

spoiling and being disposed of unused. A further benefit, which was of major importance to the

PEX, was the improvement to seed forecasting capability based on the ability of the system to

maintain up to date stock levels and demand (based on despatch data). There would also be

reduction of inventory investment and purchase costs through improved inventory monitoring.

Furthermore, the rule enforcement by the system would ensure better transaction controls e.g.

inspections were to be carried out in sequence and the system was not to allow updating of say

the harvesting stage inspection before the pre-flowering inspection had been done.

5.3.5 Design and Development

The design and development of the PEX system was outsourced to an external service provider,

whom I refer to as the developer. The developer’ s choice of methodology was Rapid Application

Developing (RAD) using prototyping. Figure 14 depicts the development process as used by the

developer firm.


87

Figure 14: Design and Development process for the PEX system

The development team from the supplier side comprised of five members and the division of

work was as shown in Table 8.

Table 8: Division of Labour for the Developer Team

Team Member Tasks

Lead Developer Analysis, Design & Quality Assurance plus project

management

Developer 1 Analysis, Design & Coding

Developer 2 (Sub-Contracted) Coding

Developer 3 Coding, User Training, Support during testing, post-

implementation support

Developer 4 Coding & Documentation

Various tools and techniques were used during the design and development i.e. a combination of

PMBOK and PRINCE2 were the project management methodologies with MS Project as the

tool; VB studio was used for developing the GUI, FastHelp was used for the Help facility, Visio

was used for documenting the data flow diagrams and entity relationship diagrams.

In the next few paragraphs I will provide a brief description of the design and development tasks.

I only include tasks that were visible to the client, even though there were other tasks such as


88

coding, unit testing, system testing, system documentation etc. that were done by the developers

independently and without the involvement of users. These tasks were carried out off-site.

Requirements Confirmation and Scoping

The first exercise that the developers engaged in was validation and confirmation of the

requirements as stipulated in the SOUR document. Though requirements had been sufficiently

documented and presented by the analyst firm, the developers needed to have their own in depth

understanding using their own tools and techniques for requirements elicitation and presentation.

This inadvertently meant that the users had to go through the whole explanation process again.

The only difference being that now not only did they use their own understanding of their work

practice – but the users were also using the analyst firm’ s representation of their work practice as

documented in the SOUR. This was done through a series of change labs with the various

functional user groups. The output from this exercise was the Requirements Confirmation

Scoping Document. The fact that there was a detailed SOUR in place ensured that less time was

spent by the developers in validating and confirming the requirements.

Prototyping

There were ten (10) prototyping sessions held even though originally there only meant to be

three (3) sessions, where the functional user groups were asked to review and provide inputs to

various aspects of the development. Initial change labs focused on the design and review of static

screens and in the latter sessions there was some logic built into the screens for process review.

The prototyping sessions were being held at the PEX premises in a room that had been dedicated

specifically for this work.

The planning, coordination and control of the prototyping sessions was the responsibility of the

developers. During the sessions we had the lead developer facilitating the sessions by taking the

users through the screens / system using a laptop and projector. There were usually two or three

other developers available to provide assistance. They kept minutes and notes of the sessions as a

record of what transpired, but also as reference for any user change requests. The users were also

given printouts of relevant screens to review and make changes. The prototyping sessions

presented learning opportunities through brainstorming and idea generation to improve the PEX

system design as well as triggering further discussion on current work practice for better


89

understanding by the developers. But initially it was difficult for the developers, as facilitators of

the sessions to get the users to engage and contribute in the brainstorming and discussions when

the analyst firm’ s team was not there.

Once the prototyping sessions had been completed, the developers proceeded to complete the

rest of the coding and development of the PEX system.

The Developed PEX System

The PEX system was custom built and was based on an n-tier architecture using MS Windows

2003 Server for the operating system, MS SQL Server 2005 for its database and MS Visual

Basic 2005 Edition for the graphical user interface.

The system design and functionality followed a modular, menu-driven design as was desired

with the core functionality provided through five modules namely, the Seed Growers

Maintenance, Crop Management, Seed Testing, Seed Processing, Seed Inventory Management,

Seed Purchasing, Seed Sales and Asset Management, Maintenance and Calibration modules.

These modules are a direct match to the PEX functions as shown in the table below (Table 9).

Table 9: PEX System Functionality

PEX Function PEX System Functionality / Module

Inspections / Basic Seed Seed Grower Maintenance

Crop Management

Seed Purchasing

Basic Seed Crop Management

Seed Laboratory Seed Testing

Seed Processing Seed Processing

Seed Inventory Management

Asset Management, Maintenance & Calibration

Seed Despatch Seed Sales


90

From a users perspective the most attractive aspect of the new system was that it was GUI-based

and it was operating in a multi-user environment, allowing users to access functionality that was

relevant to their work practice.

Once developed, tested and signed off, the PEX solution was deployed at two sites i.e. a

production site and a disaster recovery site with requisite infrastructure allowing for mirroring

and data replication between the two sites.

Training

Training was provided over a one week period and trainees were divided according to the system

functionality groups / modules which were closely aligned to the PEX functional structure. The

training sessions were formally structured and users, though sharing machines, were interacting

directly with the system during training. The training objectives were outlined as:

• To familiarise the users with the whole system

• To familiarise each user group with their specific modules

• Preparation for user acceptance testing

• Introduction and finalisation of User Guide

• To make sure that all functionality required has been incorporated into the system

On completion of the training, training evaluation forms were completed by each participant and

overall rating for the training and the trainer was said to be ‘Good’ (i.e. based on a rating scale of

Poor, Average, Good and Excellent). The conduct of the training allowed for ‘play-like’

activities and the formal manner in which the training was planned and conducted ensured that

the users were aware or ‘conscious’ of the fact that they were engaged in a learning activity.

User Acceptance Testing (UAT)

The testing was carried out once the system had been fully developed with the logic built into it.

It was carried out by the functional users who were being supported by the developer team. A

representative of the analyst firm was also available during the testing. The functional users

received one week training on the system prior to commencement of the UAT. The test plan had

been developed jointly between the users and developers. The users had to then prepare the test


91

data for use during testing. The UAT took three (3) weeks instead of the planned two weeks

because it was only during the testing that users, especially from the Lab, began to understand

what they were going to get system-wise and therefore took their time in the testing. As a result a

lot of changes were requested by the Labs during the testing.

Once the UAT was completed and signed off by the users, the product was presented to PEX

Management. The approach taken was that the functional users presented their respective

modules to Management instead of having the developers do it. Management were seeing the

system for the first time and not surprisingly, came up with a number of changes that they

wanted incorporated into the system.

Piloting

The PEX system was piloted for a period of three months before finally ‘going live’ . The

piloting was intended for end to end testing of the seed multiplication process in a ‘live’

environment using real data and not test data. All the functional user groups were involved and

encouraged to use the system for all their daily transactions. The developers provided on-site

support / handholding during this piloting phase.

5.3.6 Post-Implementation Support

The developer firm was retained by PEX to provide support and maintenance on the system for

an initial period of two years. In terms of the support, PEX had assigned one of the (junior)

developers who had been involved in the development as a dedicated support resource. But two-

three months into the support contract, the designated resource resigned from the company and

was replaced by a new COX recruit who had not been involved in the development. The support

procedure was such that PEX users would log faults / problems with the COX help desk and

depending on the severity of the fault / problem the COX would send a representative on-site to

resolve the problem.

5.3.7 Post Implementation Review

5.3.7.1 Objectives and Scope Currently conducting post implementation reviews is not part of standard practice. But in the

case of the PEX project, a PIR was carried out. The review was carried out six months after


92

‘going live’ , which was deemed to be sufficient time for the users to have started to confidently

use the system. The main review objective was to assess system effectiveness, but in light of this

research study I also added one or two questions that addressed individual impact and learning.

Table 10 summarises the scope of the review.

Table 10: PEX Project Review Scope (from PIR Report, 2008, pg 7)

Review Area Scope Liaison With

System Effectiveness • system features • system utilization • information accuracy and data

content • processing schedules • response times • security and controls

Users

System’ s compliance with requirements

• Evaluate compliance against both functional & Non-functional requirements

Users

Developers

Training and Documentation • Training effectiveness • Level and type of training

provided • Adequacy of training manuals /

documentation • Adequacy and completeness of

procedure and user manuals

Users

PEX Project Manager

Developers

Hardware Infrastructure Review

• Adequacy of processing and storage facilities

• Adequacy of disaster recovery plan and procedures

• Networking capabilities

PEX Project Manager

GIT

Support & Maintenance Review

• Evaluate adherence to the SLA’ s • Review adequacy of support

procedures

Users, PEX Project Manager

Developers

Systems development process • Methodology & Approach • Organisational Impact • Individual Impact including ‘what

has been learnt’

Users, PEX Project Manager

Developers

The data for the review, as stated earlier, was collected through written interviews based on

open-ended question formats (i.e. Appendices A, B and C) with all the users as well as the

developers. I took responsibility to administer the interviews myself and at the beginning of each

interview I explained that I would also be using the data for this research and therefore requested

for consent.


93

Findings from the review are detailed in the PEX PIR report, but for purposes of this research

study I have included some of the key findings and more specifically those that have a bearing

on learning, which is of interest to this research.

5.3.7.2 Post Implementation Review Findings System Effectiveness and Compliance to Requirements

Though the findings from the review suggested that the PEX system developed was consistent

with the user requirements and the users were confident that it addressed business and processing

requirements, even at the time of the review it had still not been fully adopted. As observed by

one of the developers ‘(we) have succeeded in providing the system, but the take-on is slow.

(some) Users take (the) system as a burden as opposed to something that will make their work

better… ’ About this the PIR Report (2008) notes:

‘It was difficult to (fully) assess system usefulness, response time, information accuracy and processing

schedule because the system is not being fully utilised. The major reasons given were that the PEX was

experiencing intermittent network problems. The other contributing factor was the lack of data, especially

as it related to current stock. There were delays in completing the stock taking exercise and capturing the

necessary data into the system. This meant that certain tasks could not be completed in the system because

of restrictions imposed by the business rules.’ (PIR Report, 2008, p. 9)

The delays in completing the stock count were attributed to some user’ s resistance to the system

and therefore not wanting to carry out their responsibilities.

Training and Documentation Findings

Though they found the training to have been good and relevant, most of the users felt that it was

too short. Again this may have been because, it was only during training and testing that users

interacted with the full system. One user when responding to a question on what they enjoyed

most about the training said: ‘Entering the real data into the system, it became much more

clearer and very relevant’ .

Support & Maintenance

The change of support personnel a month into the support contract was cited as a major problem

by users. This, as expected, initially brought about some challenges because the users were still

learning and so was the support person. Though the users, in their evaluation had said the


94

training provided had been effective – most of what was happening during the support was

training. As one of the developers observed when asked about the effectiveness of the support,

‘... (its) very frustrating because it is more of training than support... to gain understanding of

how the modules work is a struggle ...’ . This seems to suggest that even at that late stage there

were issues of user learning that could be traced back to the ISD process itself.

Systems Development Process

As stated earlier the project was originally conceptualised by the management of PEX who were

not even represented in the PIC. Some of the key project resources who were there at the

beginning when the project was conceptualised changed during the course of the project

implementation i.e. the project sponsor changed and so did the principal user and some of the

functional users. This created problems with consistency and continuity since those who were

there to drive and agree on the analysis phase were not there during the design phase. Also,

those that came in required time to learn first about the seed multiplication work activity and

then also try to catch up with the project and provide meaningful input. As one of the users

remarked ‘ I am new in the Lab (when I joined the project) so the project / system development

assisted with understanding / learning Lab work; it forced me to read so as to provide input on

new screens developed; I also learnt more about the seed multiplication business processes as a

whole.’ Out of the fourteen people interviewed post implementation, only four were involved in

the project during the SOUR phase.

The use of slightly different representation techniques, presented a challenge for the users and as

such it took quite a number of prototyping sessions before the screen designs could be finalised

and accepted. As observed by the client project manager, ‘The SOUR workflow was closer to

users so they could follow; Design stage DFD’ s (were) confusing to users until the screen

design, users could now follow especially once the logic was built-in; training also cemented

understanding’

One user suggested the lack of exposure to computers as a contributing factor to the slow

transition from the SOUR to the design & development phases ‘I understood the SOUR phase,

but it was my first exposure to computers and so I found it difficult to understand the design and

development process. I understood objectives from SOUR to Design & Development but the

problem is in depth understanding of computers’ . As pointed out by the lead developer ‘...


95

initially they were more comfortable talking to ... (the lead analyst who also happens to be the

researcher) than us ... spent a bit of time knowing each other ... learning each other .... took a

while for them to trust us ...’ . As a result of this the transition from the Analysis phase (tools and

object) to the Design phase took some time.

The prototyping sessions provided learning opportunities in three out of the four possible

learning opportunities mentioned by Bødker and Grönbeck (1996). These were in brainstorming,

idea generation and in instances where the focus was on the prototype itself i.e. that was when

during the demonstration there was a system error of some sort. This occurred mainly during the

latter parts of the prototyping sessions when some logic had been built into the system. The

learning situation presented by simulation of work-like actions using the prototype was not that

evident, mainly I would think because the prototyping approach that was used was different from

the one used by Bødker and Grönbeck i.e. a full prototype system had not been developed, but

instead static screens and minimal logic were what was used. Furthermore, the users did not have

direct interaction with the prototype – it was more of a demonstration. When asked about

whether the selected approach was a contributory factor to user learning and therefore slow

system uptake, the lead developer stated that they had deliberately chosen that approach because

in their assessment, use of a fully fledged mock system would have distracted the users and

resulted in more re-engineering work. He said ‘... we look at the people involved on the client

side ... if we think the fancy part (i.e. logic) would be distractive we leave it out ... from the

beginning we had unease about whether we would get agreement on screens ....’ He explained

that the low level of user participation during the requirements validation change lab led them to

conclude that the use of a ‘static’ prototype would be the best approach for the PEX project.

Organisational and Individual Impact

This was an important question that was included in the PIR in order to address the concerns of

this research. The following responses were given by the fourteen users and two developers that

were interviewed.


96

Table 11: Users and Developers PIR Responses on Learning

Users

User 1

‘I learnt about the value of using computers, the importance of in-depth knowledge of use of

the system. It puts Botswana on the map with regards to IT use and has resulted in

excitement and motivation to work. It will assist towards obtaining Lab accreditation.’

User 2

‘I need more time to reflect on individual impact, but it has opened up my mind to work

opportunities’

User 3

‘I was new in the Lab so project / system development assisted with understanding / learning

Lab work. It forced me to read so as to provide input on new screens developed. I also learnt

more about SMU business processes as a whole.’

User 4

‘I learnt how to use a computer and that it’ s easy to store volumes of information e.g. on a

memory stick. I also learnt how to type and also that analysis translated business processes

into work that could be automated which was beneficial’

User 5

‘Learnt how to manage records.’

User 6

‘The system could be useful if operational and could assist with data management.’

User 7

‘This was my first involvement in an IT project. I learnt the value of using the system to

support business processes. I also learnt that computers are faster and more accurate than

human beings. Furthermore, I learnt the value of computer storage and security provided

through such storage as opposed to maintaining manual documentation’

User 8

‘I learnt the ideal way of operations through automation since the system enforces business

rules. But this may also act as a constraint especially as it relates to Inspection.’

User 9

‘ I am a previous computer user, but this current system is different from those used before’


97

User10

‘I learnt how to use a computer since it was my first encounter with it. Creating Lots in the

warehouse was a new thing.’

User 11

‘I learnt how to use the system and the need to have complete information in order to process

transactions.’

User 12

‘I am still learning about computers. I learnt the need to capture data on time and follow

process flow as defined to benefit from the SMIMS implementation.’

User 13

Blank

User 14

‘I learnt how to handle people who are using computers for the first time i.e. resistant users.

Also human-computer interaction. I think younger users would have enjoyed and benefited

from being involved in the development.’

Developers

Developer 1

‘ Gained an understanding of the PEX environment and most importantly that even within

the same Ministry or government environment you may require a different implementation

approach’

Developer 2

‘Learnt how to manage clients’

The first user response suggests that the system had brought about work improvement that may

even result in accreditation of the (lab) work practice. The is the same user who only seemed to

have realised what was going on in the PEX project during testing and as a result a number of

changes had been requested to the system design. It would therefore appear from this statement,

that understanding, even if it occurred during the latter stages of the design and development was

crucial to the final outcome or artefact. I was therefore left to wonder how much more could


98

have been achieved if understanding of the process had been obtained earlier on in the project. I

also wondered whether evaluating that understanding throughout the process could be of value.

These are thoughts that I took up during the change lab sessions where, I, together with other

practitioners reflected on practice as reported on in the next chapter.

The fourth user response was from a user who was involved in the project from the analysis

phase of the project who observed that though when the project started they did not have much

understanding of the ISD process they had learnt how from analysis you could end up with a

working system which they now had.

User 5, 6 and 7 responses point to learning about specific areas of work practice that had been

improved by the new artefact, e.g. management of records and data management. It’ s learning

about the value of using computer technology to support business processes which seems to

suggest that it was something that had not been anticipated especially given that the old system

operated in a standalone environment and therefore most users did not interact with it much.

User response 8 pointed to learning about process flow and business rules that needed to be

followed as imposed by the system. The enforcement of business rules was viewed both

positively and negatively. The positive aspect was that it was now possible to ensure strict

adherence to procedures and standards. The negative side, according to the users, was that it

could be a constraint since for example with Inspections the system did not allow skipping of

stages. The response by user 9 also seems to suggest that because the PEX system offered more

than what they had used before in terms of a computer application, the learning was therefore at

a much advanced level.

The creation of Lots in the warehouse (User 10) was a business improvement brought about as a

result of the automation. Coming up with lot numbers that could be used to identify seed from

planting through to when the seed was processed, packaged and stored in the warehouse was a

major challenge throughout the project. During the prototyping sessions various numbering

systems were suggested and tried out until eventually the group agreed to one which they all felt

would work. It was an interesting learning cycle of questioning, modelling, testing, modifying

and then testing again.


99

In summary learning by users took place at different levels depending on their level of IT

literacy. For example, for new computer users, the learning was on the use of computers, their

value and potential. Whereas for users who had prior exposure to computers, learning was on

how automation can be achieved through the ISD process and on business process improvement

e.g. the lots example. Interestingly though, none of this learning as listed led to motivation for

high system utilisation and uptake. Thus the question that still remained was why despite this

reported learning there was still slow system uptake. Hence my desire to bring together IS

practitioners to review the current process and suggest improvements that would enhance

learning and system uptake.

At stated previously, during the PIR it was only possible to interview two out of the five

developers involved in the PEX project. Though the lead developer was also interviewed, this

was only done after the first change lab session and the interest then was to follow up on some of

the issues raised at the change lab. However, the two developers interviewed reported learning

mainly on how to manage clients in different environments. As developer 1 pointed out, despite

their experience in developing and implementing systems for other government departments –

the PEX project presented yet again a different environment which they needed to understand.

The slow systems uptake was also a concern for the developers and this particular developer later

stated his reflections on the project that:

‘We could have done better if we had a better understanding of the environment. More

change labs with the key users were required to match their expectations of where they

see the system fitting into their environment’

5.4 Conclusion

In this chapter I have presented a description of current ISD practice in Botswana by identifying

the key actions as described by the GITREP as well as the tasks as carried out in the case project.

The findings from the post implementation review confirm my concerns about slow systems

uptake when the actions /tasks are carried out in a manner described in this chapter. Though

there are a number of interesting factors such as the prototyping methodology used, and the

profile of the functional users that one could conclude as having contributed to slow system

uptake, I chose to leave the analysis to a wider group of users and practitioners. This way


100

whatever new ISD practice model was arrived at would have wider acceptance and applicability

within the government system. I therefore set out to engage as many practitioners and users as

was possible in the review and redesign of practice. I also wanted to establish whether this was a

problem experienced in other government ISD projects and if it was, to find a solution that could

be implemented across all government projects. In the next chapter I report on how a group of

users and developers came together in two change lab sessions to review and redesign the ISD

practice in Botswana.

Chapter 4 Research Design, Methodology & Data 62

Documents

Transcript of Chapter 4 Research Design, Methodology & Data 62