GOOD GOVERNANCE IN PUBLIC PROCUREMENT: AN EVALUATION OF

THE ROLE OF AN E-PROCUREMENT SYSTEM

by

Rajesh Kumar Shakya

CATHIE WITTY, PhD, Faculty Mentor and Chair

CHERISE MOORE, PhD, Committee Member

JOAN VERMILLION, PhD, Committee Member

Elizabeth Koenig, JD, Dean, School of Public Service Leadership

A Dissertation Presented in Partial Fulfillment

Of the Requirements for the Degree

Doctor of Public Administration

Capella University

June 2015

© Rajesh Kumar Shakya, 2015

Abstract

The outcome of public procurement reforms being carried out in most developing

countries is negligible and has not provided significant support for good governance in

the procurement sector despite having sound objectives with the initiatives. Presently,

most of the governments around the world are implementing e-government procurement

(e-GP) systems as a tool for public procurement reform for better governance in the

public procurement sector. The implementation of e-procurement has been accepted as

one of the most promising and feasible options for governments in delivering enhanced

governance in the acquisition of goods, works, and services for the public sector. This

action research analyzed the implementation of the e-GP system in a developing country

using quantitative and qualitative methods of data analysis following the utilization-

focused evaluation framework. The study focused in the utility of the e-GP system,

addressing different dimensions of good governance and emphasizing transparency,

accountability, corruption control, efficiency, effectiveness and predictability, easy

access, rule of law and equity, and civil society awareness. It studied the users’

perception about the role of the e-GP system in public procurement good governance, the

users’ attitude towards e-GP system use, and the e-GP system’s impact on good

governance. The research revealed that an e-GP system significantly enables and

enhances good governance in the public procurement sector. The research indicated that

an e-GP system can be fully leveraged for enhancing good governance if, among other

things, a comprehensive e-GP system implementation framework is followed with

adequate planning, phasing of implementation is based on the procurement cycle basis

rather than an incremental functional basis, and the e-GP system addresses all the key

aspects of good governance rather than focusing only on 1 or 2 aspects in isolation.

iv

Dedication

I dedicate this dissertation to my dear wife, Dr. Dhandu Rani Shakya; my son,

Ayush; and daughter Ayusha, without whose encouragement, strength, support, and

enduring patience, the completion of this dissertation would not have been possible. In

addition, I always feel blessings from my late dad and mom, and support from my sisters

and brother.

v

Acknowledgments

This dissertation came to this shape with the tremendous help and support of

many people. I would first like to thank my mentor and committee chair, Dr. Cathie

Witty, for all of her advice, motivation, and patience she gave me throughout the

dissertation process. I would like to extend special thanks to my committee members, Dr.

Cherise Moore and Dr. Joan Vermillion, who kept me honest and insisted that I go that

extra mile to make the dissertation precise and clear. I sincerely thank Tanka Mani

Sharma and Naresh Kumar Chapagain for facilitating environment, and Bhim Khatiwada

in the survey field test and coordinating the participants of the focus group discussions. I

also want to thank all the respondents from public entities, and procurement officers and

bidders of this study for providing valuable advice. I also want to thank several

colleagues, including V. S. Krishnakumar, Knut Leipold, S. M. Quamrul Hassan, Dr.

Paul Schapper, and Hunt La Cascia for keeping me motivated.

vi

Table of Contents

Acknowledgments v  

List of Tables ix  

List of Figures xii  

CHAPTER 1. INTRODUCTION 1  

Introduction to the Problem 1  

Background of the Study 4  

Statement of the Problem 10  

Purpose of the Study 13  

Rationale 13  

Research Questions 16  

Significance of the Study 17  

Definition of Terms 19  

Assumptions and Limitations 24  

Conceptual Framework 25  

Organization of the Remainder of the Study 32  

CHAPTER 2. LITERATURE REVIEW 34  

Brief History of Procurement 34  

Public Procurement Reforms 34  

Objectives of Public Procurement 37  

CHAPTER 3. METHODOLOGY 46  

Action Research Evaluation Approach 46  

Research Design 55  

vii

Sample 57  

Instrumentation 59  

Data Collection 61  

Data Analysis 62  

Ethical Considerations 65  

CHAPTER 4. RESULTS 68  

Site Description 68  

Research Methodology Applied to Data Collection and Analysis 69  

Analysis, Synthesis, and Findings 70  

Data Collection 84  

Survey Results 87  

Two-Way Analysis of Variance Test 95  

Focus Group Discussions 98  

Summary 114  

CHAPTER 5. DISCUSSION, IMPLICATIONS, AND RECOMMENDATIONS 116  

Revisiting the Problem 117  

Discussion of Findings, Implications, and Recommendations 118  

Implementation and Incorporation of Action Research Outcomes 134  

Conclusions 136  

Limitations of the Study 141  

Recommendations for Further Research 143  

Summary 145  

REFERENCES 147  

viii

APPENDIX A. RECOMMENDED E-GOVERNMENT PROCUREMENT SYSTEM FEATURES 158  

APPENDIX B. VALIDITY AND RELIABILITY ANALYSIS TABLES 165  

APPENDIX C. FRIEDMAN/CHI-SQUARE TEST ANALYSIS AND MULTIVARIATE ANALYSIS TABLES 176  

APPENDIX D. STUDY FEEDBACK AND RECOMMENDATIONS 188  

APPENDIX E. E-GOVERNMENT PROCUREMENT IMPLEMENTATION FRAMEWORK 193  

ix

List of Tables

Table 1. Reliability Analysis for Dimensions of Good Governance Factors 73  

Table 2. Summary of Good Governance Construct Results 84  

Table 3. Study Respondent Demographics 85  

Table 4. Two-Way Analysis of Variance Test on Research Questions and Underlying Hypotheses 96  

Table B1. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Transparency 165  

Table B2. Communalities for Transparency 166  

Table B3. Total Variance Explained for Transparency 167  

Table B4. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Accountability 168  

Table B5. Communalities for Accountability 168  

Table B6. Total Variance Explained for Accountability 169  

Table B7. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Corruption Control 169  

Table B8. Communalities for Corruption Control 170  

Table B9. Total Variance Explained for Corruption Control 170  

Table B10. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Efficiency, Effectiveness, and Predictability 170  

Table B11. Communalities for Efficiency, Effectiveness, and Predictability 171  

Table B12. Total Variance Explained for Efficiency, Effectiveness, and Predictability 171  

Table B13. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Easy Access 172  

Table B14. Communalities for Easy Access 172  

Table B15. Total Variance Explained for Easy Access 173  

x

Table B16. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Rule of Law and Equity 173  

Table B17. Communalities for Rule of Law and Equity 174  

Table B18. Total Variance Explained for Rule of Law and Equity 174  

Table B19. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Civil Society Awareness 175  

Table B20. Communalities for Civil Society Awareness 175  

Table B21. Total Variance Explained for Civil Society Awareness 175  

Table C1. Friedman Test Results for Transparency 176  

Table C2. Friedman Chi-Square for Goodness-for-Fit Test Results for Transparency 177  

Table C3. Friedman Test Results for Accountability 178  

Table C4. Friedman Chi-Square for Goodness-for-Fit Test Results for Accountability 178  

Table C5. Friedman Test Results for Corruption Control 179  

Table C6. Friedman Chi-Square for Goodness-for-Fit Test Results for Corruption Control 179  

Table C7. Friedman Test Results for Efficiency, Effectiveness, and Predictability 179  

Table C8. Friedman Chi-Square for Goodness-for-Fit Test Results for Efficiency, Effectiveness, and Predictability 180  

Table C9. Friedman Test Results for Easy Access 180  

Table C10. Friedman Chi-Square for Goodness-for-Fit Test Results for Easy Access 180  

Table C11. Friedman Test Results for Rule of Law and Equity 181  

Table C12. Friedman Chi-Square for Goodness-for-Fit Test Results for Rule of Law and Equity 181  

Table C13. Friedman Test Results for Civil Society Awareness 181  

xi

Table C14. Friedman Chi-Square for Goodness-for-Fit Test Results for Civil Society Awareness 182  

Table C15. Multivariate Analysis of Variance: Descriptive Statistics 183  

Table C16. Multivariate Analysis of Variance: Tests 184  

Table C17. Multivariate Analysis of Variance: Tests of Between-Subjects Effects 185  

xii

List of Figures

Figure 1. Objectives of e-procurement. 27  

Figure 2. Conceptual framework of study. 30  

Figure 3. Utilization-focused evaluation interaction. 48  

Figure 4. Three user groups of an e-procurement system. 58  

Figure 5. Structure of study survey questionnaire. 60  

Figure 6. Marginal means of dimensions of good governance. 97  

Figure E1. E-government procurement implementation framework. 195  

Figure E2. E-government procurement readiness assessment. 197  

1

CHAPTER 1. INTRODUCTION

Introduction to the Problem

Procurement policies establish the practice of procurement to apply methods and

public financial resources that produce the best social and economic outcomes, within a

value set that is ethical, nondiscriminatory and transparent. This establishes the objectives

of public procurement in terms of principles that includes value-for-money, transparency,

open and effective competition, nondiscrimination, accountability and due process. In the

opinion of Veiga Malta, Schapper, Calvo-Gonzalez, and Berroa (2011), these principles

describe both the ends and the means of public procurement.

The underlying concept is that good governance in public procurement sector

exists when public money is spent for the right purpose aiming to procure with the right

quality, right quantity, and right price at right time and place from the right source.

Morita and Zaelke (2005) stated that many factors play an important role in

socioeconomic development but good governance is now recognized as playing an

essential role in the achievement of sustainable socioeconomic development of a country.

The authors also agree that good governance is supported by clear decision-making

procedures at the level of public authorities’, civil society participation as well as the

ability to enforce rights and obligations through legal mechanisms.

Governance is a broad and inclusive concept that covers formal as well as

informal spheres of life in any human society, emphasizing the importance of compliance

2

with such ethical virtues as transparency, equality, fairness, responsibility, accountability,

participation, responsiveness to the needs of the people, and efficiency in the

management of public fund. Good governance refers to the process and structure that

insures good management of resources (Asian Development Bank, Inter-American, Bank,

& World Bank, 2004). Good governance promotes and requires the rule of law,

upholding such dimensions of the good governance as transparency, accountability,

participation, effectiveness, efficiency, predictability, fighting corruption, consistency,

and coherence (Kaufmann, Kraay, & Zoido-Lobaton, 1999; Organization for Economic

Co-operation and Development, 2006; United Nations Economic and Social Commission

for Asia and the Pacific, 2009).

Public procurement is one of the sectors that require attention in terms of good

governance as it is directly related to the expenditure of public funds. It is common that

e-procurement is one of the major interventions taken by many governments as an

innovative enabler for good governance in public procurement. For the purpose of this

action research, the e-procurement is operationally defined as the collaborative utilization

of information and communications technologies (especially the Internet) by government

agencies and other actors in procurement community in conducting all activities of the

government procurement process cycle for the acquisition of goods, works, and services

with good governance in public procurement (Shakya, 2012). A common government

procurement process cycle includes the processes starting from the demand aggregation,

procurement planning, selection of procurement methods, preparation of bidding

document, invitation to bids, bid submission, bid opening, bid evaluation, contract

awards, contract management, payment processing, and contract closure. Procurement

3

principles are generally provisioned through the Public Procurement Act and detailed

processes, methods, and rules are devised in the Public Procurement Regulations.

Presently, most of the governments around the world implement e-procurement

systems as a tool for public procurement reform for better governance in the public

procurement sector (Callender & Schapper, 2003; Wittig, n.d.). This action research was

rooted in the recognition that despite having sound objectives with public procurement

reform initiatives, and significant resources invested for that, the reforms are negligible in

most developing countries. Developing countries are characterized by low and middle

income (World Bank, 2013), low levels of industrialization, poor infrastructure, slow in

technology adoption, widespread illiteracy, and poor living standards among their

populations as a whole (Al Moalla & Li, 2010).

One of the reasons they are getting fewer results could be that the potentials of e-

procurement systems are not fully utilized, and the procurement sector could not obtain

significant support for a sustainable framework for good governance from the policy

level (Ladipo, Sanchez, & Sopher, 2009). Hence, this action research analyzed the

implementation of the e-procurement system in one of the developing countries in terms

of its utility addressing different dimensions of good governance by measuring the users’

perception about the role of the e-procurement in public procurement good governance,

their attitude towards its use, and finally the impact on good governance. The research

problem relates to all public development issues as public procurement deals with public

funds, and the direct impact would be for the public entities spending public money, the

bidders (goods suppliers, works contractors, service consultants) participating in

4

government procurement contracts, and regulatory/oversight agencies for regulating and

overseeing such interventions in public procurement.

The e-procurement supports the social control of public expenditure through its

different online measures of public transparency (De Almeida, 2002). Rotchanakitumnuai

(2013) supported the theory of good governance through the research that a transparent e-

procurement process has a positive effect on good governance practice, increasing cost-

effectiveness and accountability, and decreasing collusion among suppliers. This action

research will improve the e-procurement system and its implementation practices for

enhancing the good governance in public procurement sector by providing feedback and

recommendations for use through the analysis of the perception of its users and level of

utilization of e-procurement system. The outcomes of the action research may also help

other countries more successfully implementing e-procurement systems.

Background of the Study

Several seminal studies have been conducted on separate aspects of e-

procurement implementations on individual case bases. As noted in those studies, there is

lack of effective e-procurement policies and practices to meet the stated objectives

(Concha, Astudillo, Porrua, & Pimenta, 2012; Vaidya, Callender, Sajeev, & Gao, 2004).

Most of the implementations did not consider adequate efficiency gain (both in terms of

value and risks) of e-procurement implementations, and did not address the required

simultaneous changes (business process reengineering) in the organization and strategy

(Trkman & McCormack, 2010), or the lack of serious planning on performance model for

perceived impacts (Gardenal, n.d.).

5

Neupane, Soar, Vaidya, and Yong (2012) believed that the proper e-procurement

management can reduce government corruption. Lee and Lee (2012) observed that the

decisions needed for enhancing good governance are inadequate, untimely, and not based

on feedback systems, knowledge, experience, and country environment. Adequate

attention to the e-procurement has not been given because of the conflict between the

technical and political requirements, and interests of the implementing countries.

The conduct of public procurement has often reflected tensions between public expectations of high standards of governance, management requirements for performance, and overt (and covert) political influence and pressure from broader stakeholder interests such as business. In many jurisdictions there is often conflict between procurement regulations (or lack of) intended for transparency and risk mitigation versus environment designed for achievement of outcomes and broader policy objectives. (Curtin University of Technology, 2007, p. 20)

For the last 2 decades, e-procurement systems have grown rapidly in different

countries. One of the major factors that drive this trend is the political commitment by

many governments to increase public procurement transparency (European Parliament

and the Council, 2004). Most of the previous systems were developed with a focus on

efficiency, not transparency. The web-based procurement-related information portal did

not necessarily extend its functionalities to improving procurement management and

streamlining the workflow processes of procurement (Carayannisa & Popescu, 2005;

Organization for Economic Co-operation and Development, 2005).

The efficacy, utilization, and completeness of e-procurement mainly impact the

bidding communities (suppliers of goods and services, contractors), citizens, society,

government procurement agencies, and regulatory/oversight bodies.

If the perceived benefits of an e-procurement system are not achieved, the

stakeholders lose their faith in reform process and uphold the status quo in terms of

6

improving governance in the public procurement sector. If e-procurement works well, it

is expected that it decreases corruption, increases efficiency, effectiveness, and

accountability, and promotes transparency and rule of law. All bidders—large and

small—are treated equally, competitiveness is increased, major stakeholders and civil

society are well informed about public procurement, and the overall governance in the

procurement sector is enhanced.

E-Procurement and Principles of Good Governance in Public Procurement

E-procurement has proven itself to be one of the more effective and efficient tools

for bringing good governance to the procurement process in many countries, and e-

procurement initiatives have received significant support from the donor community

including the World Bank, the Asian Development Bank, the Inter-American

Development Bank, and the African Development Bank (United Nations, 2011). There is

agreement about the following core principles of good governance according to the

Organization for Economic Co-operation and Development (2006a), United Nations

Economic and Social Commission for Asia and the Pacific (2009), and the World Bank

(Kaufmann et al., 1999). These core principles of good governance—transparency,

accountability, participation, rule of law, effectiveness, efficiency, predictability, fighting

corruption, proportionality, consistency, and coherence—are also applicable to public

procurement. Public procurement is an important function of government (Thai, 2001)

and has to satisfy requirements for the procurement of goods, works, and services in a

timely manner. Furthermore, it has to meet the basic principles of good governance:

transparency, accountability, and integrity (Callender & Schapper, 2003; Wittig, n.d.).

Schapper, Veiga Malta, and Gilbert (2006) stated that in jurisdictions in which the

7

dominant political concern has been the principles of transparency, equity, and fair

dealing, management of public procurement through an extensive regulatory framework

often constitutes the status quo.

The Multilateral Development Bank, comprising the Asian Development Bank,

the Inter-American Development Bank, and the World Bank, are using an electronic

government procurement readiness assessment developed in November 2004 to conduct a

high-level review of its procurement environment and determine the level of readiness in

the country to make a smooth transition to e-procurement. The implementing countries

already feel that the instrument should be updated.

The joint World Bank and Organization for Economic Co-operation and

Development Assistance Committee Procurement Round Table initiative, developing

countries, and bilateral and multilateral donors developed a set of tools and standards—

Organization for Economic Co-operation and Development methodology—that provide

guidance for improvements in procurement systems and the results they produce

(Organization for Economic Co-operation and Development, 2006a) but does not cover

e-procurement. Similar to the Organization for Economic Co-operation and Development

Assistance Committee, the World Bank also uses the country procurement assessment

report to assist its member countries in analyzing their present procurement policies,

organization, and procedures.

Concha et al. (2012) proposed the comprehensive e-government procurement (e-

GP) observatory maturity model, which aims to satisfy four e-GP high-level

requirements, for each participant agency:

8

• Identifying its current state of maturity and capability in an integral, multi-aspect manner

• Comparing itself with other governments evaluated with the same model

• Building a feasible improvement roadmap to improve its levels of capability and maturity

• Informing policymakers about its readiness for integration with other e-government initiatives, and defining improvement programs when required

The e-GP observatory maturity model provides the guidance only for the start-up

activities to be carried out by the governments before implementation of the e-GP system.

This action research considered the e-GP observatory maturity model while developing

an e-GP implementation framework, which was one of the outcomes of this research.

Country Focus on the Aspects of Good Governance

There are many possible factors contributing to the issue of good governance.

Governments in developing countries are taking e-procurement as an introduction of

information and communications technologies in the government. Such intended or

unintended focus taken by the governments in implementing e-procurement systems

could be the reason of fewer outcomes (Ladipo et al., 2009). Another possible factor may

be the lack of clear policy of the government on the use and practice of e-procurement.

Asiimbwe (2012) reported news citing the observation of a procurement expert in

Uganda, saying, “Lack of clear policies is hindering the use of e-procurement in Uganda”

(p. 1). She noted that it is a challenge to build and sustain relationships with entities,

which are not prepared to go online, and small businesses not enabled for e-procurement.

On the other hand, the Government of Bangladesh has issued a Guideline for e-

Government Procurement clearly explaining the policy as well as procedures and

9

practices for the use of e-GP. As the result of such clarity, the current e-procurement

implementation trend has experienced an upsurge beyond projections, which is resulting

in a more positive procurement environment.

Curtin University of Technology (2007) pointed that many countries are focusing

merely on cost saving by the use of e-procurement than their focus on enhancing overall

good governance. The author indicated another problem may arise from the deployment

of e-procurement system without adequate business process reengineering of the

traditional procurement processes and practices before introducing into the e-

procurement. One of the reasons could be that “little attention has been paid to the need

for periodical measurement of the e-GP systems’ performance and goal satisfaction”

(Concha et al., 2012, p. 550) as well as not considering the e-procurement in systemic

manner as a mainstream tool which can enhance the comprehensive dimensions of good

governance in procurement sector. Curtin University of Technology (2007) listed the key

lessons learned by most of the countries, which relate to ensuring

• Effective government leadership and management of the implementation plan • Comprehensive user buyer and manager awareness and training so they are

confident to use the systems and their initial concerns have been addressed • Supplier awareness and training must be addressed. • The potential e-procurement environment is assessed and the information used

in developing the implementation plan. • The transition from paper- to electronic-based processes is planned and

supports the need to change the process. • Legislative support, especially identified by South American countries (p. 27)

Therefore, this action research project contributes to the improvement of

employee organizational behavior as well as stakeholders involved in public procurement

process. This action research establishes the importance of taking a systemic or holistic

approach when introducing e-procurement system taking adequate measures to include

10

and support all aspects of good governance that seamlessly link with the principles of

public procurement. The action research follows a utilization-focused evaluation of e-

procurement implementation in one of the developing countries using a contribution

analysis framework (Patton, 2012b), to analyze the country-based lessons on e-

procurement as they relate to implementation around the world.

Statement of the Problem

Not only in the Public Procurement Monitoring and Regulatory Agency of the

research site country but also in any other countries, there is no evidence of academic

studies or practitioner evaluation that have attempted to assess the role of an e-

procurement system in enhancing good governance in the public procurement sector by

applying an integrated and comprehensive model of research including the application of

good governance standards. Additionally, there is incomplete knowledge of the

underlying attitudes, perceptions, and knowledge of any dimensions that could have an

impact on good governance for the public procurement sector.

If good governance in the procurement sector is not effective, it could lead to poor

strategic country development as most nations spend about 20% of gross domestic

product on public procurement (Callender & Mathews, 2000; R. Carter & Grimm, 2001),

and developing nations spend up to 50% (Schiavo-Campo & Sundaram, 2000, p. 315).

E-procurement in the site country was started by the comprehensive study by the

Asia Foundation (Shakya, 2003). This study focused on feasibility and challenges, and

prepared a roadmap for e-GP implementation in the country. An e-GP readiness

assessment conducted in 2007, even before the procurement monitoring and regulatory

agency was established, assigned the country a reasonable score of 2.22 out of 4 in level

11

of readiness. The major influence on this score was the lack of an appropriate lead agency

to drive and sponsor this procurement reform, according to the World Bank; however,

several building blocks for the implementation of e-procurement in the country are

already in place.

Internet connectivity is available in almost all government administrative

premises up to local level, and a significant number of rural community centers, cyber-

cafes, and business centers are established across the country with information and

communications technologies facilities. Currently, the public procurement and

monitoring agency is established and strongly fulfilling its mandate of governing public

procurement sector in the country including the introduction of a unitary national e-

procurement system. By this time, the journey of e-procurement in the country has come

a long way, but still needs a clear strategy to take the initiative forward.

In an earlier discussion, Shakya (2003) noted that a “lack of transparency in

government procurement has been a systemic problem in public procurement reform. In

addition, inefficiency due to manual and paper-driven procurement processes has resulted

in poor service and low sales volume” (p. 54). The study found that establishing an e-

procurement system could help improve transparency, efficiency, and value-for-money in

government procurement. Stakeholders (government agencies and suppliers) showed

their support for the e-procurement concept following this study, stating that

The government should capitalize this concept and encourage community to go in the direction of e-procurement. The strategy should recognize that there are elements of risk, which are unavoidable. If the full benefits of e-procurement are to be realized it should be managed very cautiously. The implementation of the e-procurement strategy will face a number of potential barriers and challenges. To help the change processes take place, key areas of concern have been identified and ways to address those potential issues noted. (Shakya, 2003, p. 54)

12

The public procurement regulatory and monitoring agency, as per the mandate of

the 2007 Public Procurement Act and 2008 Public Procurement Regulations, started the

phased development of centralized e-GP system. Phase I (from the procurement process

of preparing annual procurement plan up to e-bid submission functions) of this system

were already rolled out in 2012; however, there were gaps, which are addressed by the

Phase II (with the functionalities of e-bid opening, e-evaluation, e-payment, contract

management, and procurement management information system) recently being

implemented.

The e-procurement system in the country is conceived as an Internet-based single,

central, and collaborative government procurement platform for all the public

procurements by the government. All the stakeholders in the procurement process get

appropriate access using their secured user name and access codes, and will have private

secured working dashboards. Direct users of the system are all the public procuring

entities, bidders, and the public procurement regulatory and monitoring agency and other

government oversight entities, tertiary service providers enabling services of e-

procurement, and maintenance service providers and the general public.

Presently, most of the governments around the world are implementing the e-GP

systems as a tool for public procurement reform for better governance in this sector

(Callender & Schapper, 2003; Wittig, n.d.). The research problem explored in this study

is rooted in the fact that despite having sound objectives of public procurement reform

initiatives with significant resources investment, the outcome of reforms in most of the

developing countries is negligible as noted earlier, because the potential of e-GP systems

are not fully used and do not have significant support for a good governance framework

13

(Ladipo et al., 2009). This action research studied and analyzed the full utilization of the

e-GP system in the country in terms of its role in enhancing good governance, and

included all available different functions, processes, practices and available technologies.

Purpose of the Study

The purpose of this research was focused on evaluating the utilization of the e-

procurement system that contributes to good governance in public procurement

(emphasizing compliance with ethical virtues on public procurement as transparency,

accountability, corruption control, efficiency, effectiveness and predictability, easy

access, rule of law and equity, and civil society awareness) in public procurement.

Findings will enhance improvements in the e-procurement system use and its

implementation to deliver better public service through e-procurement.

Rationale

There is a vacuum of research-based knowledge base on the role of e-GP in terms

of addressing good governance agenda of the public procurement reform. Chvalkovská

and Skuhrovec (2010) explored the Transparency Index in public organizations as an

influence of the e-procurement implementation. Ermal and Miranda (2012) studied the

efficiency of the Albanian e-procurement system in the reduction of corruption in public

procurement practices. A Curtin University of Technology (2007) Multilateral

Development Bank survey—only one survey so far on e-procurement systems in

Asia/Oceania, South America, and Europe—was carried out as part of a project of the

Multilateral Development Bank Harmonization of e-Procurement Group. This survey

aimed only to provide information for jurisdictions that were at the inception stage of

14

implementing e-procurement and was designed to provide information on e-tendering, e-

reverse auctioning, and e-purchasing systems in relation to

• System functionality achieved

• The governance, business, management, and technical issues that were addressed in the transition to e-procurement

• The costs of development or acquisition, and implementation of the systems and the annual costs to operate and maintain the systems

• Benefits recognized by stakeholders

• Outcomes and outcome measures

• Success factors and lessons learnt

• Other observations as appropriate

The Curtin University of Technology (2007) Multilateral Development Bank

survey did not study any areas of goal satisfaction on good governance aimed by the

public procurement reform initiatives and the contribution of e-procurement in better

governance in public procurement.

The reasons to implement e-procurement are to achieve one or the other

combinations of maximum good governance objectives in public procurement. Despite

having sound objectives of public procurement reform initiatives, the tools and

instruments are not sometimes properly designed to meet such objectives. Although an e-

GP system is quickly becoming an essential agenda of national e-government programs,

because it enables active transparency and favors efficient vendor relationships, little

attention has been paid to the need for periodical measurement of e-procurement portals’

performance and goal satisfaction (Concha et al., 2012). It is now imperative that e-

procurement systems are analyzed to find out whether adequate measures were taken in

15

their design and whether the desired goals of good governance in public procurement

initiatives have been achieved.

This research chose to focus on the good governance aspect of the e-procurement

system in one of the developing countries, because this issue is especially important in

the developing countries, in which public funds are significantly abused under the cover

of public procurement. Using good governance as the focus for the study provided

meaningful insight into possible problems with the implementation e-procurement

systems in the country. The main site for the research is the public procurement

regulatory and monitoring agency of the site country.

There have not been other studies on the study of perception on the role of e-

procurement systems in contributing to good governance in an integrated system.

Therefore, results gained through a study done in the site country will strengthen and

facilitate research on e-procurement systems in other countries with similar situations.

This action research supports improving the accountability of government

officers, efficiency in the procurement process, reduction of corruption, and creation a

favorable environment for bidders to participate in government businesses (contracts)

with trust and confidence; it also promotes the enhancement of good governance in

public procurement by analyzing the gaps in current implementation and practice.

These outcomes are possible through an ongoing collaborative process between

the research facilitator and the program stakeholders working within their focus groups to

evaluate the program’s resources, gather data, evaluate findings and execute the

recommendations (Patton, 2012a).

16

Through the utilization-focused evaluation survey and focus group discussions on

the e-procurement implementation, changes will emerge as stakeholders connect with the

challenges in the system, providing them with clear vision for change and improvement

in e-procurement service operations geared to long-term good governance.

Research Questions

The seminal researchers over the last 15 years have shown that the introduction of

some forms of e-procurement has helped improve transparency, efficiency, and cost

saving, while reducing corruption in public procurement. Nevertheless, despite the

proliferation of literature touching on theory and practice, most of the research focused

on some influences within the general behavior of e-procurement systems. Therefore, in

spite of all of these studies focused on separate aspects of the system, there is a gap in

studies considering e-procurement as a mainstream and integrated tool to reform public

procurement and enhance good governance in the public procurement sector. To achieve

good governance, it is important to identify the perceptions of the stakeholders working

within the e-procurement system and their contributions to good governance in all levels

of the total system to finally assess the best practices of e-GP.

This research analyzes the perception of the key stakeholders along a variety of

dimensions of good governance working within the operating e-procurement system.

The key research questions were

1. How useful is the e-procurement system in enhancing good governance in public procurement?

2. Does e-procurement add value to public procurement?

17

A survey was conducted to collect data under seven sets of questions to assess

perceptions about the relationship e-procurement within the operational objective

dimensions of good governance: transparency; accountability; corruption control;

efficiency, effectiveness and predictability; easy access; rule of law and equity; and civil

society awareness.

The survey also collected demographic data of the respondents such as age group,

respondent type, years of e-procurement use, and annual participation in e-procurement.

The survey was designed to identify the best practices e-procurement governance and the

required measures of intervention in e-procurement systems for the benefits of good

governance.

Significance of the Study

The study of the role of e-procurement system in enhancing good governance in

the public procurement sector holds an immense significance in e-procurement domain

globally. The results of the study should provide insights and guiding framework for the

public procurement regulatory and monitoring agency of the site country as well as

global public procurement community in case of the perception, attitude, and knowledge

to improve the e-procurement implementation. The study analyses current setting of

practices, gaps, challenges in an e-procurement system, and provides insights on

improvements to be made to overcome the good-governance-related inconsistencies.

In order for research to provide meaningful outcomes, the researcher analyzed the

available theoretical concepts and international practices. The following aspects represent

the key significance of the study:

18

• Integrated approach of good governance factors coverage in e-procurement system implementation—The significance of this study outstands from the other studies is based on its approach to see the problems from the point of view of contribution of e-procurement system in enhancing good governance in totality. It does not attribute individual factors in isolation for the good governance in public procurement. The simple reason for taking an integrated approach is that if e-procurement is implemented just for the enhancement of transparency, or efficiency, or cost saving in silos, then remaining fundamental principles of public procurement will have to be addressed in traditional manner. All the procurement processes and factors of good governance are strongly interlinked. Inadequate focus on each of key governance factors imbalances the public procurement sector, and practices become inefficient. This study analyzes the impact of fragmented implementation of the e-procurement system in the country, and intends to establish a theory that a comprehensive and integrated approach of e-procurement system implementation enhances good governance in the public procurement sector.

• Redefined approach of phasing to cover procurement cycle in e-procurement system implementation—This action research redefines the phased implementation of e-procurement system. Despite many scholar literatures recommend phased implementation of e-GP implementation as one of the essential requirement for the success of e-GP system, it appeared from the research that the phased approach of implementations were mainly focused on incremental approach through the implementation of facilities step by step by improving the website functions: (a) provide procurement-related information through an information portal on procurement, (b) publish procurement notice, (c) publish procurement notice with bidding document download option, (d) publish contract awards, (e) allow to upload bid document in PDF format, (f) publish annual procurement plan, (g) provide facility of e-bidding, (h) monitor procurement performance through electronic procurement information management system in-built in e-GP, and (i) provide basic contract management functions.

The e-procurement system implementation approach without having implemented complete set of key processes is lengthy and raises significant amount of rework moving from one stage to another. This approach is not favorable for achieving desired target. This research establishes the theory that if phase implementation is planned, the phase should be designed in such a way that it demonstrates the effectiveness of implementation and integrity of the process of procurements. Phasing is not effective if few functions are done electronically and other functions of the process are done in manual system. Functions and tools in e-procurement system should be available to complete one procurement logical cycle.

19

• Result-oriented action research based on utilization-focused evaluation—This action research is based on utilization-focused evaluation and follows the process recommended by Patton (2012a). Significance of the study is reflected in the corrective actions taken by the public procurement regulatory and monitoring agency in the site country in Phase 2 of the e-procurement implementation.

The chosen country for the research spends 60% of its total annual budget in public procurement, which highlights the urgency for good governance in public procurement.

The government of the country has initiated the implementation of a national level e-procurement system as one of its interventions for the good governance in public procurement sector. The regulatory and monitoring agency is the implementing agency for the pilot implementation of the first phase of e-procurement system that began in May 2013. The pilot involved 13 nominated agencies. The outcome of the project and the utilization-focused evaluation enabled the public procurement regulatory and monitoring agency, and other intended users to improve the implementation and use of e-procurement system “by designing an intervention to create change” (Patton, 2012a, p. 57). According to Ramirez and Broadhead (2013), “[utilization-focused evaluation] evaluators facilitate a learning process with attention to how real people in the real world might apply evaluation findings and experiences. In designing a utilization-focused evaluation the attention is constantly on the intended use by intended users” (p. 1). As the action is the only way to generate and test new social science knowledge, the evaluation findings in participation of intended users were prepared as the classified recommendations for use. The regulatory and monitoring agency has prioritized the research evaluation recommendations under Strategic and Operational categories so that the use of findings becomes more manageable. Most of the recommendations have been implemented in the rollout version of the second phase of e-procurement implementations in the country.

• Contribution to international community—One of the outcomes of this action research is the e-GP implementation framework. The e-GP implementation framework is developed based on the research, lessons learned in the site country for the research and other countries, and international best practices. The e-GP implementation framework and the evaluation framework can be used as a model for other countries.

Definition of Terms

The definitions of terms discussed as follows are presented to show how these

terms are defined within the scope of this study:

20

Accountability. Accountability means that officials are responsible for the actions

and decisions that they take in relation to procurement and for the resulting outcomes,

and secondly that their actions and decisions are transparent (Commonwealth of

Australia, Department of Finance and Deregulation, Business Procurement and Asset

Management Group, 2013).

Civil society awareness. To ensure accountability, transparency mechanisms such

as public oversight and monitoring, are important. Civil society can play an important

role in raising awareness around public procurement and other areas of public financial

management (Jeppesen, 2010). According to Kpundeh (2004), the aim of this awareness

raising is to exert political pressure and pressure to the public officials to use public

resources only for the benefits of citizens, not for their private purposes (p. 259).

Corruption control. Corruption in public procurement relates to the awarding of

public procurement contracts to a preselected supplier to the detriment of the public

interest through violation of the principles of competition for the purpose of gaining

personal benefit (Pashev, Dyulgerov, & Kaschiev, 2006). Of all government activities,

public procurement is most vulnerable to corruption.

The favoritism with regard to a specific supplier, which is harmful to the public interest, can be observed anywhere from the smallest day-to-day supplies to the biggest tendering procedures directly supervised and controlled by senior government officials. The personal benefit can take the form of cash, power, jobs in the private or public sector, and so on. (p. 5)

Easy access. Building the capacity of bidders has been identified as one of the

success factors of public procurement reforms. Many bidders are limited in various

capacity issues including lack of basic knowledge in the law, inadequate capacity to

appreciate the standard tender documents, poor access to tender information and

21

insufficient technical and managerial skills to be competitive in the tendering process

(Osei-Tutu, Mensah, & Ameyaw, 2011). Easy access to procurement information

empowers the bidders, and can provide quality and competitive bids to the government.

Effectiveness. Effectiveness relates to how well outcomes meet objectives. It

concerns the immediate characteristics of an agency’s outputs, especially in terms of

price, quality and quantity, and the degree to which outputs contribute to specified

outcomes (The Australian National Audit Office, 2010).

Effectiveness can be achieved by ensuring that the property or service being

sought will make the maximum possible contribution to the relevant outcome. This

entails correctly identifying the need, accurately drafting functional specifications,

rigorously assessing responses and negotiating the final contract, and then diligently

managing the contract. Comprehensive monitoring and assessment at all stages of the

procurement process further contributes to effectiveness (The Australian National Audit

Office, 2010).

Efficiency. Efficiency relates to the productivity of the resources used to conduct

an activity in order to achieve the maximum value for the resources used. In relation to

procurement, it includes the selection of a procurement process that is consistent with

government policy and is the most appropriate to the procurement objective under the

prevailing circumstances (The Australian National Audit Office, 2010).

Efficiency in procurement is enhanced by conducting transparent, fair and

appropriately competitive processes of a scale commensurate with the size and risk

profile of each particular project.

22

E-government. E-government is a form of e-business in governance and refers to

the processes and structures needed to deliver electronic services to the public (citizens

and businesses), collaborate with business partners and to conduct electronic transactions

within an organizational entity (Backus, 2001).

E-government procurement (e-GP)/e-procurement.

There are various definitions, which often hinge on the difference between e-procurement and e-tendering. In the private sector, e-procurement is often used to mean the technology that supports the postcontract award purchase-to-pay or transactional cycle. E-tendering is then used to describe the technology used around the precontract award phase, the category management or strategic sourcing cycle - supplier selection, tendering, and contract management. (Smith, n.d., p. 5)

E-procurement is operationally defined as the collaborative utilization of

information and communications technologies (especially the Internet) by government

agencies and other actors in procurement community in conducting all activities of the

government procurement process cycle for the acquisition of goods, works, and services

(Shakya, 2012).

Equity. Equity promotes nondiscrimination. All potential suppliers to government

must, subject to procurement rules, be treated equitably based on their commercial, legal,

technical and financial abilities and not be discriminated against due to their size, degree

of foreign affiliation or ownership, location, or the origin of their goods and services

(Australian Government Department of Finance, 2014).

Good governance. Good governance promotes the rule of law, upholding the

dimensions of the good governance, such as transparency, accountability, participation,

effectiveness, efficiency, predictability, fighting corruption, consistency, and coherence

23

(Kaufmann et al., 1999; Organization for Economic Co-operation and Development,

2006a; United Nations Economic and Social Commission for Asia and the Pacific, 2009).

Governance. A wide and inclusive concept that covers formal as well as informal

spheres of life in any human society, emphasizing compliance with those ethical virtues

as transparency, equality, fairness, responsibility, accountability, participation,

responsiveness to the needs of the people, and efficiency in the management of public

funds.

Predictability. Standardization always helps in ensuring fair competition,

reducing abuse and improving predictability in procurement outcomes (Vajpey, 2010).

Procurement planning and transparency enhances the predictability of the procurement

process, so that the bidders can plan their resources beforehand (Lynch, n.d.).

Public entities/procuring entities. Public entities are the government agencies,

departments and units, authorized to spend public money for the procurement of goods,

works and services. In some countries, procuring entities are called contracting

authorities, procuring agencies, public entities, or purchasing agencies. For the sake of

this study, the terms procuring entities and public entities are used interchangeably.

Rule of law. Many institutions identify a fair, impartial, and accessible justice

system and a representative government as key element of the rule of law. In this study,

the term rule of law is applied for the enforcement of the Public Procurement Act and

Public Procurement Regulations to strengthen the compliance across all the public

procurements the fair and equitably, equally, consistently, coherently, and prospectively

to all stakeholders (Morita & Zaelke, 2005).

24

Transparency. Transparency is a primary consideration throughout the

procurement process. The general principle is that sufficient and relevant information

must be made available to all interested parties consistently and in a timely manner

through a readily accessible, widely available medium at no or reasonable cost

(Australian Government Department of Finance, 2014). This general principle is

applicable to all aspects of government procurements. Transparency is established when

all aspects of procurement in an entity are available for appropriate

scrutiny/review/audit/reference.

Value-for-money. Value-for-money is the core principle underpinning public

procurement. Value-for-money is a comparison of relevant benefits and costs on a whole

of life basis. Value-for-money can be achieved by encouraging competition in

procurement bidding among bidders, promoting the efficient, effective, economical, and

ethical use of resources, through highest level of accountability and transparency,

encourage appropriate engagement with risks and being commensurate with the scale and

scope of the business requirement (Australian Government Department of Finance, 2014;

Commonwealth of Australia, Department of Finance and Deregulation, Business

Procurement and Asset Management Group, 2013). Procurement practices and

procedures must be directed to achieving the best available value-for-money in the

acquisition of goods and services to deliver, or support delivery of, government

programs.

Assumptions and Limitations

This action research was conducted to assess the initial stage of e-procurement

system implementation and the creation of good governance standards in the site country

25

chosen for this research through the survey of stakeholders involved in public

procurement activities in the country. As a result, the findings from the surveys may not

be generalizable to other globally strategic implementations of e-procurement systems.

Additionally, implementation approaches are different in different countries, and this also

limits the generalization of any findings to larger strategic levels. The research effort

focused on sampled number of participants to gather perceptual data. The resultant data

on perception could be different if the e-procurement system implementation is assessed

later, in a few years.

Conceptual Framework

There is no consensus on what constitutes good governance and that is not a

purpose of this study. Siddiquee and Mohamed (2007) stated that

Good governance involves much more than efficient management of economic and social resources or improved delivery of public services; it essentially relates to the quality of relationship between the government and the governed. It also involves relationships and cooperation among the public sector, private agencies and civil society organizations in policy processes as well as the delivery of services. (p. 288)

Researchers study the implementation of e-procurement in the public sector from

two different perspectives. The majority of research focuses on success factors of the e-

procurement implementation and another as an intervention for reform. This research

falls under the second stream. Considerable research attention has focused on the linkage

between e-procurement and corruption reduction, the relationship between cost saving

and e-procurement, the potential of e-procurement to improve transparency and

efficiency in public procurement, and e-procurement and accountability.

26

For the purpose of this study, however, the terms of good governance are used

after analyzing the considerable body of scholarship available in public procurement and

e-procurement sector, and their implementations. The researcher believes good

governance in public procurement can be achieved only with the balanced contribution of

different factors addressing the core principles of procurement. The theory that this

researcher establishes here puts focus on an integrated model, which covers the whole

spectrum of dimensions of good governance in public procurement. The dimensions

under this study include (a) transparency; (b) accountability; (c) corruption control; (d)

efficiency, effectiveness, and predictability; (e) easy access; (f) rule of law and equity;

and (g) civil society awareness.

E-GP, or e-procurement, is more than just a government website on the Internet to

publish information about the procurement opportunities and other related information.

The strategic objectives of e-procurement (see Figure 1) are to enhance good governance

in public procurement sector, obtain value-for-money, and socioeconomic development

in the country (Shakya, 2014). The good governance aspect of the e-procurement

supports and simplifies governance for all stakeholders involved in public procurement:

public/procuring entities, suppliers from business, oversight and regulatory authorities,

and citizens.

The use of the e-procurement system can connect all stakeholders and support

processes for online procurement transactions and other related activities. In other words,

e-GP supports and stimulates good governance. Therefore, the objectives of e-

procurement run parallel to the objectives of good governance. Implementation of e-

procurement system for the good governance can be seen as an exercise of economic,

27

political, and administrative authority to better manage the public procurement sector in

the electronic environment.

Figure 1. Objectives of e-procurement.

The conceptual framework of this study is based on few theoretical concepts.

Based on the investigation of the literature, the research findings revealed that generic e-

government applications and e-procurement-related research largely used the theories of

the technology acceptance model, diffusion of innovations theory, unified theory of

acceptance and use of technology, and theory of planned behavior to justify the relevance

and results of the implementation of e-services. These theories tested for the e-

procurement implementations and assessing benefits also have given results consistent

28

with acceptable variance across various studies. Although a large number of theories and

theoretical constructs were borrowed from the reference disciplines, their utilization by e-

procurement researchers appears to be largely random in approach.

Rogers (2003) suggested that technological innovation will be adopted smoothly

and diffuse faster if it possesses five attributes: relative advantage, triability,

observability, complexity, and compatibility. Studies of innovation adoption have been

using these attributes amongst others in their search for factors influencing technological

innovation adoption. In case of this study, e-procurement as an innovative tool for reform

in public procurement sector is required to be adopted by its stakeholders, mainly by the

procuring agencies, bidders, and regulatory/oversight agencies.

Because of its adoption and diffusion in the public procurement sector, it is

expected to enhance good governance in the sector through its relative advantages such

as increased transparency, transparency, efficiency, effectiveness, predictability in

procurement and procurement opportunities, equal treatment to the bidders, compliance

to the acts and regulations by enforcing rule of law through the behavioral changes in

carrying out the procurement processes. Most of such e-procurement initiations are

initially pilot implemented (triability) to confirm the observability. After the successful

trial of the e-procurement system implementation, it should be diffused across the similar

procuring agencies with dynamic sustainable adoption model. The e-procurement

systems should be easy to access and should not marginalize any prospective bidders

giving equal competitiveness and provide user-friendly user interface and simplicity to

understand avoiding complexity of the classical procurement processes and practices

after adequate business process reengineering. Despite the comprehensive business

29

process reengineering, the e-procurement system should still be compliant and maintain

its compatibility with the prevalent Public Procurement Act, Public Procurement

Regulations, and guidelines.

As stated previously, this study evaluated whether the use of the e-GP as a

technological innovation contributes to good governance in the procurement sector of the

government of the country chosen for the research. The public procurement regulatory

and monitoring agency of the country has already completed the trial run of the e-

procurement system, developed in compliance with the Public Procurement Act and

Public Procurement Regulations, and currently at the phase of nationwide diffusion

(rollout implementation) and stakeholder adoption.

The level of coordinated incorporation, integration of public procurement, and

different dimensions of good governance in process and practices of the e-procurement

system development and implementation influence the role of e-procurement in

enhancing good governance. However, having all the best practices and dimensions of

good governance in a single system may require considerable understanding of the

principles of good governance in public procurement. The model developed in this study

for accessing the role of e-procurement is focused on enhancing good governance in

public procurement sector generates a comprehensive integrated framework for the

governments implementing e-procurement system.

The perceptions of three groups of stakeholders were assessed: procuring entities,

bidders, and regulatory/oversight agencies. The attitudes of responsible executives

regulating and oversight agencies, procuring entity executives and procurement

professional, emphasizing good governance commitment are also important (Ararat &

30

Ugur, 2003). Public procurement professionals and bidding community have a major

influence on enhancing good governance (Hui, Othman, Omar, Rahman, & Haron, 2011).

Strong governance commitment is required in order to implement an appropriate E-GP

system that ensures transparency (Leipold, 2007).

Research Framework

A conceptual research framework is designed with seven constructs supported by

70 nonparametric questions (see Figure 2). The survey also collected demographic data

of the respondents by four parametric questions (i.e., age group, respondent type, year of

e-procurement use, and their annual participation in e-procurement).

Figure 2. Conceptual framework of study.

31

A solid conceptual good governance framework was needed for this research, and

is embedded in e-procurement system framework for the good governance in public

procurement underlying this research. Each one of these operational variables of the good

governance framework, which derives its theoretical and conceptual underpinning of the

e-procurement system, has been taken as central to the good governance in public

procurement. The main hypotheses of the study were

• H0: The e-procurement system does not significantly contribute to enhance good governance in the public procurement sector.

• HA: The e-procurement system significantly contributes to enhance good governance in the public procurement sector.

Other hypotheses were put forth as

Transparency.

• H10: The e-procurement system does not significantly contribute to enhance transparency of procurement processes and information in public procurement.

• H1A: The e-procurement system significantly contributes to enhance transparency of procurement processes and information in public procurement.

Accountability.

• H20: The e-procurement system does not significantly contribute to enhance accountability of the actions in public procurement.

• H2A: The e-procurement system significantly contributes to enhance accountability of the actions in public procurement.

Corruption control.

• H30: The e-procurement system does not significantly contribute to enhance corruption control and reduce corruption in public procurement.

• H3A: The e-procurement system significantly contributes to enhance corruption control and reduce corruption in public procurement.

32

Efficiency, effectiveness, and predictability. • H40: The e-procurement system does not significantly contribute to enhance

efficiency, effectiveness, and predictability in public procurement.

• H4A: The e-procurement system significantly contributes to enhance efficiency, effectiveness, and predictability in public procurement.

Easy access.

• H50: The e-procurement system does not significantly contribute to enhance easy access to government procurement opportunities.

• H5A: The e-procurement system significantly contributes to enhance easy access to government procurement opportunities.

Rule of law and equity.

• H60: The e-procurement system does not significantly contribute to enhance rule of law in compliance with the act and regulations and equity in treating bidders in public procurement. H6A: The e-procurement system does not significantly contribute to enhance rule of law in compliance with the act and regulations and equity in treating bidders in public procurement.

Civil society.

• H70: The e-procurement system does not significantly contribute to enhance civil society awareness about public procurement.

• H7A: The e-procurement system significantly contributes to enhance civil society awareness about public procurement.

Organization of the Remainder of the Study

The remainder of this dissertation is divided into four chapters. Chapter 2

comprises the extensive literature review surrounding public procurement, reform in

public procurement, the role of e-procurement to enhance the good governance, and its

different dimensions in public procurement sector. It starts with a broad discussion of

33

governance, good governance, and its relatively new concepts around e-governance and

then the good governance through e-procurement.

Chapter 3 outlines the overall research design and methodology for the remainder

of the study. It states the specific hypotheses that the survey instrument sought to validate

or reject. It includes the specific methodologies that the researcher used to gather

quantitative data to support or reject these hypotheses. Included in Chapter 3 are the

specific questions that supported the constructs of each one of the hypotheses outlined.

Finally, the remainder of Chapter 3 outlines the data collection methodologies in order to

support the data analysis and findings in Chapters 4 and 5.

34

CHAPTER 2. LITERATURE REVIEW

Brief History of Procurement

Procurement as a separate area of scholarly study is not very old. Still today in

many countries, procurement (purchasing) is considered a clerical function in a finance or

accounts unit. According to Quesada, González, Mueller, and Mueller (2010), only in the

1970s did purchasing/procurement began to receive academic attention and recognition

as one of the key administrative functions.

It was Porter’s (1980) seminal work, however, that prompted private corporate firms to think of procurement as a strategic function rather than simply and administrative one. Since the 1980s, procurement has evolved from being viewed as merely a process for buying goods and services for a firm, to being more comprehensively defined as all the activities necessary to acquire goods and services needed to achieve user requirements. (Tassabehji & Moorhouse, 2008, p. 517)

The strategic importance of procurement has been reiterated frequently, and is still one of the critical themes found in the literature (Drake & Lee, 2009; Ellram & Carr, 1994; Kocabasoglu, 2002; Ordanini & Rubera, 2008; Rajagopal & Bernard, 1993; Rink & Fox, 1999). Soares-Aguíar and Palma-Dos-Reis (2008) and Drake and Lee (2009) argued the importance of giving procurement a strategic role in the organization and agreed that achieving world-class status in procurement requires leadership and alignment of purchasing strategy with business strategy. Although relatively fewer studies have analyzed procurement and its impact on different functional, firm, or supply chain performance objectives (Croom & Brandon-Jones, 2007), components of the basic e-procurement model can be gleaned from contributions in the literature at both the strategic and operational levels. (Quesada et al., 2010, p. 516)

Public Procurement Reforms

The history of public procurement reform is not very old. Focusing on the

analysis of different procurement systems, their merits and demerits, challenges of

35

implementations, purpose of public procurement reforms, and construction of laws

favoring actual reforms and their implementations, very little literature, especially the

academic papers, have been published. Building a body of public procurement

knowledge, one of attributes of a profession, is very critical (Thai, 2004). The body of

knowledge focusing on the comprehensive study on e-government procurement (e-GP)

and its contribution in enhancing good governance in the public procurement sector is

very limited.

Despite its widespread appeal and popularity, governance and more specifically

good governance has remained a highly contested term. Ever since its formulation in a

World Bank document in 1989, various parties used the term governance to refer to a

wide range of issues, ideas, and policies. Early proponents such as the World Bank

(1992) defined governance as “the manner in which the power is exercised in the

management of a country’s social and economic resources for development” (p. 1).

Governance involves the process of decision making and the formal and informal

structures that are set to get and implement the decision (United Nations Economic and

Social Commission for Asia and the Pacific, 2007). Typically the government sector has

a problem with transparency in procurement (Mitra & Gupta, 2007). In this sector, there

are both formal and informal government structures. Formal government structures make

decisions on procurement and how they are implemented. Informal decision making

involves kitchen cabinets, or informal advisors. Such informal decision making is

frequently the result of corrupt practices or leads to corrupt practices. Good governance

requires a fair process of transactions and services with accountable administration (Bedi,

Singh, & Srivastava, 2001; Saxena, 2006). In general, corruption can be business or

36

government related. Economic corruption is the use of public resources for private gains.

Political corruption is the violation of the formal rules governing the allocation of public

resources by administrators for financial gains or political support (Ampratwum, 2008).

E-GP can provide a solution to address the problems of the lack of transparency and

accountability, efficiency, effectiveness and predictability, corruption control, rule of law

and ethics, easy access to procurement opportunities, bureaucratic inefficiency and

ineffectiveness, nepotism, and cronyism.

According to Arrowsmith, Treumer, Fejø, and Jiang (n.d.), the last decade of the

20th century witnessed the start of a global revolution in the regulation of public

procurement. Actually, challenges in procurement reforms are beyond procurement

regulations to include procurement process, methods, procurement organizational

structure, and workforce. According to Thai (2001), procurement reforms occur

constantly in all countries, developed as well as developing countries. The purpose and

scope of reforms may vary from country to country depending on the problems in the

country, but fundamental principles of procurement are in general same either that is in

the developing country or in the developed country. The evolution of procurement is

ongoing, from bureaucratic, manual process, green procurement to e-procurement

systems. Therefore, the efforts have been made for the reform activities for the good

governance in key sectors.

Turning to actual public procurement reforms, procurement laws, and their

implementation, only a small literature has emerged on public procurement practices.

Few studies attempt cross-country comparisons of reform experiences or procurement

practices—exceptions being Trepte (1998) for Bhutan and Laos, and Wittig (1998) for

37

South Africa and Uganda. Moreover, only a few of the articles attempt any form of

impact assessment, benchmarking, quantification of the costs, benefits, or effects of

existing procurement practices or reform initiatives.

Further, the sheer magnitude of resources devoted to public procurement compels

attention. Most nations spend about 20% of gross domestic product on public

procurement (Callender & Mathews, 2000; R. Carter & Grimm, 2001), and developing

nations spend up to 50% (Schiavo-Campo & Sundaram, 2000, p. 315). In the United

States, federal public procurement during 2009 accounted for over $534 billion (General

Services Administration, n.d.) and over 13% of the total federal budget; of the U.S.

states’ annual budgets, roughly half goes toward goods and services procured from the

private sector (Knight, Caldwell, Harland, & Telgen, 2003).

According to Schapper (2008), civil society awareness and oversight of

procurement processes can enhance governance from the demand side. The author

recommended using e-procurement for enhancing transparency and civil awareness.

Schapper stated,

E-procurement has a range of potential benefits for both process and transparency. Procurement of goods, works, and services through Internet-based information technologies (e-procurement) is emerging worldwide with the potential to reform processes, improve market access, and promote integrity in public procurement. However, rather than a technical solution, e-procurement should be considered part of procurement reform in general. (p. 115)

Objectives of Public Procurement

Good governance, value-for-money, and socioeconomic development are the

objectives for procurement (Shakya, 2014). It should be noted that these objectives are

focused on outputs and results rather than procedures.

38

The public procurement system manages up to 20% of the gross domestic product

(Organization for Economic Co-operation and Development, 2006b). A system that

performs well in terms of productivity, efficiency, and integrity has wide-ranging

national benefits. On the other hand, weaknesses in procurement management under

deliver social services, increase sovereign risk for foreign investment, and reduce

national economic growth (International Monetary Fund, 2010). The urgency of

achieving high performance standards in public procurement has been emphasized by the

International Monetary Fund (1998), Asia-Pacific Economic Cooperation (1999), and the

Organization for Economic Co-operation and Development (2006a), which can be

established only through the establishment of best practice in terms of the application of

methods and resources that produce the best social and economic outcomes from public

financial resources, within a value framework of good governance.

Another source of information is the country procurement assessment reports

prepared by the World Bank and other development agencies. These reports provide a

diagnostic assessment of the strengths and weaknesses of the procurement systems in

individual developing countries. The country procurement assessment reports are

publicly available through the World Bank website. According to Evenett and Hoekman

(2005), the country procurement assessment report process has led to recommendations

and financing of action plans to modernize or reform existing procurement systems in a

number of countries. As is true of the more academic literature, the impact of this work

on the performance of national public procurement systems has not been well

documented, in the sense that the country procurement assessment reports tend to focus

primarily on legal compliance—whether laws and required procedures have been

39

implemented and procurement acts, rules, regulations, are complied with by the

stakeholders.

There is some literature available that talks about the contribution of e-

procurement on different aspects of the good governance separately or in part. There are

also differences in defining the good governance in public procurement sector. Most of

the scholars and researchers have limited the definition of good governance (Grindle,

2010). It is found that similar conclusions are drawn in more recent literature as well.

Without much consideration countries are adopting the same line of thoughts while

defining the e-procurement strategy or in the implementation of the e-procurement

systems. Research and findings cover a wide range of good governance aspects: E-

procurement can add service value and increase cost savings to the government (Casaki

& Gelleri, 2005; Iqbal & Seo, 2008; Rai, Tang, Brown, & Keil, 2006); E-procurement

can improve transparency and governance change business practice and encourage new

suppliers/vendors to participate in public procurement (Harris & Rajora, 2006); E-

procurement is an effective system which achieves good governance in procurement and

limits political interference (Heywood, 2002); The face-to-face interaction between

government and bidding community can encourage favoritism and bribery in

procurement (Hui et al., 2011); E-procurement highly emphasize on the automatic

enforcement of regulations and reduces the face-to-face interaction and hence prevents

abuse and fraud (Rotchanakitumnuai, 2010); and so on.

The researcher reviewed the body of works in e-procurement that proves the

claim of this researcher that e-procurement is still treated just as one of the tools for

mainstream good governance in public procurement sector, which means the full

40

potential of e-procurement system is still not leveraged by the governments around the

world.

A review of the existing literature showed that e-procurement in the countries

implemented following adequate planning produced increased strategic and competitive

advantages while at the same time contributing to the good governance and also saving

millions of dollars in public procurement and operational costs. Additionally, the

literature reviewed in e-procurement implementation discussed different success factors

at the macro level. Implementation approach plays significant role in the success of e-GP

system. Deise, Nowikow, King, and Wright (2000) supported the implementation of an e-

procurement system covering complete procurement cycle for achieving benefits from

the e-procurement system. However, there is no literature, which discusses and analyzes

the perception of the integrated comprehensive role of e-GP to enhance the good

governance in public procurement in a true and measurable sense. Ng’ambi and Brown

(2004) also agreed with Suchman (1977) that in social research, no event has a single

cause and each event has multiple effects. No single factor is a necessary and sufficient

cause of any other factor (Ng’ambi & Brown, 2004).

As stated by Carlson and O’Neal-McElrath (2008), research and evaluation

determine the impact, effectiveness, and efficiency of any program, in this case the

implementation of an e-GP system as one of the instruments for establishing good

governance in the public procurement sector. The Australian Government Department of

Finance (n.d.) asserted from its experience in implementing e-procurement system that by

communicating e-procurement objectives and strategy and understanding suppliers’

concerns, it is possible to ensure suppliers’ continued support. Getting stakeholders to

41

participate can be difficult, especially if they do not understand the value of the

implementation to themselves and their functional area.

The outcomes of the study will impact the overall perspective on the e-GP

implementation by the governments by highlighting and informing user of the importance

of focusing on the different aspects of good governance, rather than minimally focusing

on transparency, or efficiency, or saving costs and one or two aspects.

According to a study by Hidayanto et al. (2012) on e-GP implementation impacts,

the transparency factor of e-procurements is expected to reduce corruption in government

projects, in an effort to achieve good corporate governance. The study by Hidayanto et al.

further concluded that the implementation of e-procurement had significant impacts on

the total cost of acquisitions, organizational characteristics, governance structure,

presourcing business processes, and procurement quotations.

The outcome of this study reveals a change in behavior and attitude towards the e-

GP system implementation by addressing gaps in different aspects of governance e-GP

implementation to meet the fundamental requirements of good governance. As stated by

Carlson and O’Neal-McElrath (2008), research and evaluation determine the impact,

effectiveness, and efficiency of any program, in this case the implementation of an e-GP

system as one of the instruments for establishing good governance in the public

procurement sector. Enhancing good governance in public procurement sector through

the effective use of e-procurement system has been appreciated by most of the countries

using e-procurement system (Organization for Economic Co-operation and Development,

n.d.). In addition to the perception assessment, the action research questions in this

research were intended to assess the opinion of the participants regarding the adequacy of

42

measures taken in the design and implementation of e-GP systems based on the best

practices in different countries. A few of the questions are designed to assess whether the

e-GP system is in compliance with the Public Procurement Act and Public Procurement

Regulations of the country chosen for this research. The findings of the evaluation

provide input for the public procurement regulatory and monitoring agency to allow them

to calibrate the existing approach of implementing e-GP system as well as provide a

comprehensive e-GP system implementation framework for the countries in the process

of implementing e-GP systems for enhancing good governance in public procurement.

The bidding community appreciates e-procurement when it reduces complex data

entry procedure, validates the data as it is entered by improving data quality, standardizes

and streamlines their work, makes the procurement processes transparent, treats equally

to all bidders, and provides provision of debriefing the reason of failure to win the

contracts, and the access to e-procurement system is hassle free. The Australian

Government Department of Finance (n.d.) asserted from its experience in implementing

e-procurement system that by communicating e-procurement objectives and strategy and

understanding suppliers’ concerns, it is possible to ensure suppliers’ continued support.

Getting stakeholders to participate can be difficult, especially if they do not understand

the value of the implementation to themselves and their functional area. This action

research also revealed that there is considerable need to communicate the utilization of an

e-GP system to enhance the good governance in public procurement to the stakeholders.

The United Nations Educational, Scientific and Cultural Organization (2005) indicated

that the e-awareness amongst the government employees and their willingness to

embrace change shall play a key role in the whole process.

43

The action research aspect of the research will improve current practice through

the application of findings generated by the utilization-focused evaluation (Patton,

2012a). Action research has been described as a form of community-based research

focusing on a participatory method of investigation (Stringer, 2007). Determining

whether the program theory will precipitate change through knowledge acquisition,

which will generate behavior and attitude change will be the focus of the utilization-

focused evaluation. Both components of the research, the theory of change model and

action research will provide the stakeholders with the opportunity to work together to

determine not only the program’s design and planning, but also program implementation

and evaluation in terms of expected measurable outcomes, whether immediate,

intermediate, or long-term (Kaufman et al., 2006).

In this research, the technology acceptance model, diffusion of innovations

theory, and e-government theory are applied to model the constructs. Saidi and Yared

(2002) asserted in support of e-government theory that e-government, as a major

instrument for achieving good governance, accompanying important investment in

media, information, and communication technologies can be an important source of

productivity growth and economic growth along with economic development and

democracy. The Organization for Economic Co-operation and Development (2006a), in

its study of country system in e-procurement in Italy, North Africa, and Egypt, asserted

that the e-government, of which e-procurement is an integral part, is one of the most

valuable information-and-communications-technologies-based administrative innovation

reforms; it can render public policy more effective, efficient and responsive to social and

political needs. It can provide a framework for organizing comprehensive e-reform,

44

starting with government and the public sector. In this study, it was assessed what

contribution of this technological innovation and diffusion bring to enhance good

governance in the public procurement sector.

For the conceptual framework, the diffusion of innovations theory also plays

significant importance, which was developed by a sociologist to explain how an

innovation diffuses through a society (Rogers, 2003). It has been used extensively to

explain the adoption of information technology innovations in an organization or society

(L. Carter & Bélanger, 2005). Innovation is “an idea, a practice, or object that is

perceived as new by an individual or another unit of adoption” (Rogers, 2003, p. 229),

and diffusion is “the process by which an innovation is communicated through certain

channels over time among the members of a social system” (Rogers, 2003, p. 229). The

rate of diffusion of innovation is affected by five attributes (Rogers, 2003): relative

advantage (“the degree to which an innovation is perceived as better than the idea it

supersedes” [p. 229]), compatibility (“the degree to which an innovation is perceived as

consistent with the existing values, past experiences, and needs of potential adopter” [p.

229]), complexity (“the degree to which an innovation is perceived as relatively difficult

to understand and to use” [p. 229]), triability (“the degree to which an innovation may be

experimented with on a limited basis” [p. 229]), and observability (“the degree to which

the results of an innovation are visible to others” [p. 229]). These characteristics are key

influences on adoption behavior (Agarwal & Prasad, 1997).

In the case of statistical analysis, Cheng and Wang (2006) stated that the Kaiser–

Meyer–Olkin measure of sampling adequacy is an index for comparing the magnitudes of

the observed correlation coefficients to the magnitudes of the partial correlation

45

coefficients. The authors suggest that large values for the Kaiser–Meyer–Olkin measure

indicate that a factor analysis of the variables is a good idea. Another indicator of the

strength of the relationship among variables is Bartlett’s test of sphericity. Foulger (1979)

argued that there is no meaning of rejecting the null hypothesis, if the observed

significance level very small. In addition to the Kaiser–Meyer–Olkin and Bartlett’s test,

other factor analysis results considered for each construct were the total variance tables,

component matrices tables, and commonalities tables.

46

CHAPTER 3. METHODOLOGY

Action Research Evaluation Approach

A utilization-focused evaluation (U-FE) approach was used for this action

research as it provides a well-suited framework (Patton, 2012a) for carrying out the

research and analysis in which people in the real world apply evaluation findings and

experiences to their work. According to Ramirez and Broadhead (2013), “U-FE

evaluators facilitate a learning process with attention to how real people in the real world

might apply evaluation findings and experiences. In designing a U-FE, the attention is

constantly on the intended use by intended users” (p. 1). As the action is the only way to

generate and test new social science knowledge, the evaluation findings in participation

of intended users will be prepared as the classified recommendations for use.

The U-FE framework (see Figure 3) covers 17 steps outlined by Patton (2012a).

This action research project adopted 15 steps in a 17-step process to find solutions or to

make recommendations to the implementing agency for the best utility of the e-

procurement system contributing to enhance the good governance in procurement sector.

Step 16, Follow-up with the users to facilitate use, and Step 17, Utilization-focused meta-

evaluation, are left for the future research. The study analyzed the implementation of e-

procurement system in one of the developing countries addressing different dimensions

of good governance. Initially, perception about the role of the e-procurement in public

procurement good governance and their attitude towards its use were assessed and

47

analyzed, and recommendations for improvement were provided. The outcome

contributes to change in the behavior and attitudes of public organizations as well as all

stakeholders in using the e-procurement for the enhancement in different aspects of good

governance in practice.

Patton’s (2012a) U-FE of complex, dynamic and adaptive systems through 15

steps out of 17 steps were the principle framework for the action research and acted as the

action plan. However, these 15 steps were incorporated and configured under Stringer’s

(2007) model of Look, Think, and Act to serve as the action plan as described as follows.

Look: Building the Picture

Step 1: Access organization readiness. U-FE is premised on evaluation

processes that are designed, implemented, and utilized by carefully selected users who

take ownership over the evaluation process by active involvement throughout the process.

For this action to take place the organizations involved need to be ready. Step 1 assessed

the organizational readiness of the chosen site for the study to take on implementing an e-

procurement system. The users’ readiness to participate, own, and learn from the

evaluation is a necessary condition for a successful outcome. The researcher conducted a

meeting to introduce the principles of U-FE concept, conducted a brief interaction with

the authority of the public procurement regulatory and monitoring agency as an

assessment of the readiness to take on responsibility for evaluation and embrace it as a

learning process. The agency authority showed keen interest and was ready. The agency

welcomed the researcher for carrying out the evaluation, and expressed to extend full

support for the facilitations required.

48

Utilization-Focused Evaluation Checklist 19

Figure 3. Utilization-focused evaluation interaction. Note. From Essentials of Utilization-Focused Evaluation (p. 426), by M. Q. Patton, 2012, Thousand Oaks, CA: Sage.

Copyright 2012 by M. Q. Patton. Reprinted with permission.

The document review was carried out which consisted of records of the registered

bidders, procuring entities, and oversight agencies in e-procurement system, reports from

earlier studies on e-procurement readiness assessment, procurement management

49

information system reports, e-procurement system usage statistical analytics, and reports

of Transparency International. Assessment of stakeholder readiness was taken place

through their formal approval from the key leaders (agency officers, procuring entities

representatives, bidders, and oversight agencies) for the participation in the evaluation.

The value of the evaluation was clearly communicated to the stakeholders. This

commitment was assured with the introductory letter sent to the stakeholders, which

described the purpose of the research, including the need for consistency in participation

throughout the project. As suggested in Ramirez and Broadhead (2013), the following

aspects were ensured in Step 1:

• A redefined role for an evaluator with emphasis on facilitation

• An investment in time and resources to build U-FE capacity particularly at the start of an evaluation process

• A clear commitment to the exercise by the leadership and primary users (staff) to allocate their time to the work

Step 2: Access evaluator readiness. This step required a self-assessment by

evaluators of their readiness and capability. Facilitation skills, commitment to U-FE, and

willingness to learn U-FE together with users are essential for the researcher. Ramirez

and Broadhead (2013) noted that self-assessment of a process that is learned by doing is

difficult to achieve. The authors further suggested to having an evaluation mentor, so that

such a reflection can take place as the mentor and the evaluator interact. In this action

research the dissertation mentor played such the evaluation mentor role.

Step 3: Engage primary intended users. Selection of primary intended users

was done using the identification criteria listed in Patton (2012a, p. 408) that included

identifying stakeholders who have a personal/vested interest or intended beneficiaries—

50

public procuring entities (in some countries, they are called contracting authorities) who

spend public money, the bidding community (suppliers and contractors, also called

economic operators in some countries), and regulatory/oversight agencies.

Step 4: Conduct situation analysis. Situational analysis was conducted jointly

with primary intended users. According to Ramirez and Broadhead (2013), “evaluation

use is people and context dependent. Use is likely to be enhanced when the evaluation

takes into account and is adapted to specific situational and organizational factors” (p.

32). A focused, intentional, and systematic situational analysis calls for ensuring the

primary intended users’ understanding of the program, appreciating stakeholders’

interests and potential areas of conflict, understanding the prior history and experiences

with evaluation, and making explicit the decision context that will affect use of decisions

and evaluation findings. The analysis may help by identifying and examining factors that

may contribute to use and those that may be barriers (Patton, 2012a). The researcher

identified barriers and limitations, such as time, funding, and potential stakeholder

turnover; and maintain flexibility in order to adapt to systems changes, if and when they

occur.

Steps 5 through 8: Step 5—Identification of primary intended uses; Step 6—

Focusing the evaluation design; Step 7—Evaluation design; and Step 8—Check the

adequacy of evaluation inquiry being addressed. The researcher worked with the

stakeholders to help determine and clarify the primary purpose of the U-FE and how the

results/recommendations can be implemented. This was done through communication

and interviews with the primary users. The researcher guided the stakeholders through

51

the process with the goal of increasing their knowledge, skills, and sense of ownership of

the e-procurement program.

The researcher allowed the stakeholders to decide on the direction of the

questions to be asked, without evaluator bias or influence. Finally, the researcher

established the evaluability assessment to meet credibility, relevance, and validity

requirements by creating outcomes evaluation in terms of the program. Questions asked

included (a) Is the program ready for evaluation? (b) Have the goals of the program been

met—that is, do the stakeholders and users have the measurable skills for using e-

procurement system? (c) How have the stakeholders been changed by the study? (d) Are

there any unanticipated outcomes from the program and the study? (e) Are best practices

being followed? and (f) How are those defined?

Tests for establishing construct validity (through using multiple sources of

evidence), internal validity (using pattern matching and logic models), and external

validity (using theory), and reliability (using focus groups discussions) were also carried

out. The researcher used multiple sources of evidence from the data, interviews of

stakeholders through questionnaires and focus groups and feedback. Use of theoretical

frameworks (as described previously) and literature reviews assisted in establishing

external validity. In summary, this stage involved

• Planning through setting priorities, ranking them in order of importance, and assigning tasks as an action plan

• Implementing the action plan by assignment

• Evaluating progress

• Checking for validity, reliability, propriety, and accuracy

52

Step 9: Theory of change work. The researcher helped stakeholders

conceptualize the evaluation design and measurement approach by helping design and

map the e-government procurement (e-GP) functions and procedures in place with the

aspects of good governance requirements like transparency, accountability, participation,

the rule of law, effectiveness, efficiency, predictability, fighting corruption,

proportionality, consistency and coherence (Organization for Economic Co-operation and

Development, 2006a). In other words, stakeholders understood the interaction of outside

factors that may impact the outcomes of the e-GP system as asserted by Weiss (2000) on

the general program theory that posits that access to information and clear

communication about the program will change knowledge, which will change behavior.

Steps 10 through 13: Step 10—Negotiate methods; Step 11—Methods

debates; Step 12—Simulate use of findings; Step 13—Gather data with ongoing

intention to use. Following Patton (2012a), who suggested involving primary intended

users in decisions regarding the selection of research design, measurement, and data

collection methods, the researcher involved the stakeholders in the data collection and

analysis process in order to generate useful findings as well as create ownership of the

problem. It enhanced the transparency and capacity and their support on the outcomes of

the evaluation. With the consent from the stakeholder quantitative data collection

technique was determined from the survey questionnaire and qualitative data from the

focus group discussions. Data collection occurred through a questionnaire survey and

focus group discussions with the stakeholders as the primary data sources, and secondary

data were collected from the different publicly available earlier project reports, study

reports, research papers, data available from Transparency International, Organization for

53

Economic Co-operation and Development, the World Bank, Asian Development Bank,

and other development agencies. In focus group protocol (Johnson & Christensen, 2008),

initial questions were general, followed by more specific open-ended questions.

Information from research literature were widely used and included.

Think: Interpreting and Analyzing

Step 14: Data presented for user engagement. The researcher presented data in

a usable format for the stakeholders by organizing the findings by tables, graphics, theme,

distinguishing facts from opinion, and establishing coding methods to maintain

anonymity of responses as well as ethical standards. In other words, data were

categorized and coded, and emerging themes were identified. This occurred by

comparing units of information (obtained through data collection), triangulating the data

through focus group discussion, and looking for recurring regularities (Merriam, 1998).

These categories reflected the purpose of the research, were exhaustive, mutually

exclusive, sensitizing and conceptually congruent (Merriam, 1998). The number of

categories was determined by the focus of the research.

The facilitator actively involved the stakeholders in triangulating the data to

facilitate data interpretation. This process involved setting meeting with the public

procurement regulatory and monitoring agency officers and identified converging as well

as diverging perspectives; and to interpreted and analyzed the data in order to complete

this cycle of the study. Recommendations for using the findings were created bearing in

mind relevance to the purpose of the study, as well as actions that are within the control

and influence of the stakeholders (Patton, 2012a).

54

Act: Resolving Problems—Planning and Implementing Sustainable Solutions

Step 15: Report produced. The researcher worked with the agency officers to

identify priorities for action. A short-term strategy with an action plan was created to

address the recommendations in the second phase of e-procurement system

implementation. The strategy and action plan included a list of goals (in order of

importance), objectives, tasks, people responsible for carrying out the action, where and

when the action will take place, as well as the resources needed to fulfill the action plan.

This included discussing with the procuring entities and bidders, and regulators the

optimum ways to disseminate the findings of the evaluation to be incorporated in the

second phase of e-GP system, and to reach all audiences in order to increase knowledge

and understanding of the e-GP system; and developing methods to follow up on project

implementation to keep it an ongoing process.

The scope of the study does not cover the final two steps, 16 and 17, of the U-FE,

although Step 16 was assessed partially to confirm whether the findings and

recommendations of the study has been considered for the improvement. As Step 16

should cover the follow-up with users to use and Step 17 should reassure the evaluation

through the consequent meta-evaluation of the evaluation itself, substantial interval of

time is required to carry out the meaningful accomplishment of these steps. Thorough

evaluation of Steps 16 and 17 are intentionally left out for future research.

Meta-Evaluation of Use

The meta-evaluation purpose is to provide stakeholders the ability to

independently assess the strengths and weaknesses of the evaluation, and guide these

stakeholders in judging the credibility of the said evaluation (Patton, 2012a). Outcomes

55

evaluation, which will focus on the changes that occurred as a result of the program

(Fitzpatrick, Sanders, & Worthen, 2011), will assess whether the governance situation in

the procurement process changed, and delivery of service improved. Outcome indicators

to measure the success of the program will be assessed through procurement performance

indicators analyzed from the e-procurement statistical analytics, reports from

procurement management information system, study, review, and evaluation reports.

The primary purpose of the meta-evaluation will be to demonstrate utility,

feasibility and accuracy of the U-FE and case study of the impact of e-procurement

system implementation on good governance. The primary intended users of the meta-

evaluation will be the legislators, development partners, oversight agencies and policy

makers of the country who influence funding as well as future implementation of e-

procurement systems in different countries, the public procurement regulatory and

monitoring agency and procuring entities who use the e-procurement system for

streamlining their procurement processes addressing the requirements of good

governance, and bidding communities (suppliers and contractors of goods, works, and

services), citizens and civil society who are impacted by the service delivery through the

use of e-procurement system. At the core of the U-FE is to provide primary intended

users with information about the program in order to reduce uncertainty of their actions

and increase their control (Morgan, 1996).

Research Design

This section presents a description of the design of this research, the methodology

that was utilized, the target population that was included, the instrumentation that was

56

implemented, how the data were collected and analyzed, and the ethical considerations

related to this research.

The findings, results, discussions, and conclusions of this research are based on

the convergent parallel mixed method approach. Both qualitative and quantitative data

collected are analyzed in order to compare results. Both types of data formats were used

in order to provide strength to the research in terms of validity and credibility. Surveys

provide quantitative data, which was then compared with the qualitative data obtained

from focus groups interviews.

The rationale for using the convergent parallel mixed method design for the

project evaluation is to obtain a comprehensive understanding of the program from both

sets of data through simultaneous comparison and interpretation. According to Patton

(2012b), U-FE is inherently participatory and collaborative in actively involving primary

intended users in all aspects of the evaluation. The participant-oriented strategy employed

using the framework of the convergent parallel mixed method approach during the data

analysis and evaluation phases.

The evaluation involved the stakeholders who examined the data from both

quantitative and qualitative sources in order to compare and triangulate the information

for accuracy and credibility. Stufflebeam’s (1999) checklist was used to serve as a

blueprint for evaluation. Evaluation methods based on these factors included

• Survey questionnaire response of suppliers and contractors of goods, works and services, citizens, society, governments procurement agencies, development agencies, and regulatory/oversight bodies and the policymakers (quantitative)

• Three focus group discussion with bidding community, procuring entities, and regulatory/oversight agencies (qualitative)

57

• Documents and records obtained from previous statistics and studies (quantitative and qualitative)

No participants were used in the analysis for which parental permission was

required.

Sample

Participants in the target population included the personnel from the three key

stakeholder groups. Each one of these participants had significant role to impact on the

good governance in public procurement. Surveying the entire population would be

difficult, if not possible to perform. However, noted statisticians Cooper and Schindler

(2008) argued that research quality is often better by using sampling versus the entire

population. Additionally, sampling is done at a lower cost and has a quicker turnaround

time.

The sample size not only was determined by the population size but also was

dependent on the specific context of the research site selected. The country in which this

research is conducted is in early stage of e-procurement implementation and the user base

is still limited. There are three key user groups of e-procurement systems (see Figure 4).

58

Figure 4. Three user groups of an e-procurement system.

Sample selection contexts are discussed as follows:

• Public entities having experience in e-procurement system—There were 29 public agencies (as of March 17, 2014) registered in the e-procurement system, out of which only 15 (almost 50%) have experience using the e-procurement system for the procurement transactions. All of those 15 public agencies were selected for the survey. Participation requests were sent to three persons in each of those agencies requesting to be responded by three levels of personnel involved in the e-procurement process: (a) Official having authority to make decision on the procurement, (b) personnel involved in the procurement process, and (c) e-procurement focal person responsible for technical aspects of the e-procurement. Each of the procuring entities registered in the e-procurement system was asked to nominate one focal point for their entities.

59

• Bidders—There were 200 private firms registered as the bidders (goods suppliers, works contractors, service consultants) in e-procurement system as of March 17, 2014, and among them, 79 bidders had experience participating in e-procurement. From among this group, 20% of the bidders (i.e., 16 bidders) having e-procurement experience were randomly selected from the e-procurement bidders database for the participation in the survey.

• Regulatory/oversight agency—Four regulatory/oversight agencies were identified in relation to public procurement in the country. Among the four, the public procurement regulatory and monitoring agency of the site country is the host of the e-procurement system and also regulatory authority for the public procurement in the country. The agency accepted to be the site for carrying out the research. Also, three other agencies were identified as the oversight agencies and were nominated for the participation in the survey.

Instrumentation

A 5-point Likert-scale survey was designed with a common set of questionnaire

with seven constructs supported by 70 nonparametric questions (see Figure 5). The

questions were designed to test the perception of three target groups (i.e., public entities,

bidders, and regulatory/oversight agencies) on the role of e-procurement under the

different dimensions of good governance: (a) transparency; (b) accountability; (c)

corruption control; (d) efficiency, effectiveness, and predictability; (e) easy access; (f)

rule of law and equity; and (g) civil society awareness. The survey also collects

demographic data of the respondents by four parametric questions (i.e., age group,

respondent type, year of e-procurement use, and their annual participation in the e-

procurement).

60

Figure 5. Structure of study survey questionnaire.

At first, a consultation meeting was held with the officers of the public

procurement regulatory and monitoring agency of the research site country as part of the

U-FE Step 7 (evaluation design) to conduct a field test to help ensure the questions are

rightly phrased, wording was clarified, Public Procurement Act and Public Procurement

Regulations requirements are covered, international best practices are covered along with

the removal or rewording of ambiguous statements and that fully supports the intended

study.

Next, a focus group discussion interview guide was designed for conducting focus

group discussions with the three separate groups. The focus group discussion interview

61

guide protocol consisted of the ground rules for the discussion as well as the guiding key

questions for driving focus group discussions to validate the participants’ reasoning

behind their views expressed through the perception survey responses, as well as to

understand how the ideas and opinions in case of good governance in public procurement

through e-procurement consolidates in the course of discussion. The site was identified,

taken approval, and the survey and the interview guide were approved by the Capella

University Institutional Review Board before engaging with participants.

Data Collection

The data were collected using the survey questionnaire designed covering the

seven dimensions of good governance in public procurement sector. The Likert range

was Strongly disagree, Disagree, Don’t know, Agree, and Strongly agree, which equated

numerals from 1 to 5 respectively. The survey questionnaire was sent to 67 respondents

under three groups of e-procurement users (i.e., public entities, bidders, and

regulatory/oversight agencies) through e-mail as well as hand-delivered. The responses to

the survey questionnaire were received through e-mails as well as in-person hand-

returned. Among them, 35 responses were from the public entity users, 12 responses from

bidders, and 6 responses from the regulatory/oversight agency users were received. This

provided a response rate of 79%.

The data collected through the questionnaire for all the seven dimensions of good

governance in public procurement through e-procurement was analyzed using measures

of central tendency which included the most common measures of mode, median, and the

mean (Bernard, 2000). Each individual response was collected and coded in Microsoft

62

Excel file. Identity of the respondents was made anonymous just using the respondent

codes.

Next, three focus group discussions with the three separate groups were

conducted to validate the participants’ reasoning behind their views expressed in

perception questionnaire survey, as well as to understand how the ideas and opinions in

case of good governance in public procurement through e-procurement consolidates in

the course of discussion. Candid opinions expressed on each of the dimensions of good

governance in relation to the use of e-procurement and its role were captured to validate

the survey responses. A total of 19 people participated in the three focus groups.

Data Analysis

Data were collected using Likert scales for the quantitative data analysis. Based

on the statistical literature review and advice of experts, Cronbach’s alpha for ensuring

reliability and factor analysis for validity were done by the Kaiser–Meyer–Olkin measure

of sampling adequacy test and Bartlett’s test of sphericity tests. Additionally both the

Friedman chi-square test and the chi-square goodness-of-fit test for nonparametric

samples were used during hypothesis testing. These tests were considered appropriate for

discrete distributions or when the distribution is not known ahead of time. Garson (2006)

stated that a nonparametric test by using the chi-square test ascertains significance if the

sample size is large enough. When there different arguments on the appropriate size

required for testing hypothesis, Garson also recommended that at the significance level of

.05, a minimum of 50 samples are necessary in order to conduct proper hypothesis

testing. This research collected 53 samples in order to be more conservative and more

robust in the data analysis section.

63

These data were then analyzed using measures of central tendency, which

included the most common measures of mode, median, and the mean (Bernard, 2000),

factor analysis. Many literatures recommended using an analysis of variance test when

there are three or more groups for comparison. The two-way analysis of variance test was

conducted in order to determine if the research questions and the underlying hypotheses

stated were supported or not supported as well as to see the significance of the

interactions between dimensions (constructs or factors). These tests were conducted on

sample data collected under with seven constructs supported by 70 nonparametric

questions from three groups of e-procurement users of public entities, bidders, and

regulatory/oversight agencies involved in both procurement transactions and e-

procurement system management. As the research model is based on multiple dependent

variables (dimensions of good governance) and independent variable divided into three

groups of users, a general linear-model-based multivariate analysis of variance was

carried out to look for an effect of one or more independent variables (user groups) on

these dependent variables at the same time (Cook, 2010).

The exact number of registered public agencies, and bidders was known from the

e-procurement system database, and the regulatory/oversight agencies were identified in

discussion with the public procurement regulatory and monitoring agency during U-FE

preparation discussions and situational analysis activity.

The responses to the survey questionnaire were received through e-mails as well

as in-person hand-returned. Among them, 35 responses were from the public entity users,

12 responses from bidders, and 6 responses from the regulatory/oversight agency users

were received. Eight out of 43 public entity users, 1 out of 7 regulatory/oversight agency

64

user, and 6 out of 17 bidders did not provide their responses. As such, 35 public entity

respondents would represent a sampling of 40.22% of the population, 12 bidder

respondents would represent a sampling of 15% of the population, and 6

regulatory/oversight respondents would represent a sampling of 85%, a reasonably

adequate and acceptable sample, especially when using nonparametric statistical

sampling (Cooper & Schindler, 2008).

The sample data were tested using chi-square goodness-for-fit. This is a

nonparametric statistical inference test that can be used in tests involving nominal data

such as a Likert-scale survey (Cooper & Schindler, 2008). It allows the researcher to test

differences “between observed distribution of data and the expected distribution based on

the null hypothesis” (Cooper & Schindler, 2008, p. 484). Based on the number of

dimension (factors) of good governance through e-procurement, the observed number of

cases in each factor of the Likert-scale survey, and the expected number of cases of the

factors based on the null hypothesis, the researcher will be able to reject or fail to reject

each one of the null hypotheses, using a .05 level of significance.

Reliability and Validity

In order for the collected data to be useful for statistical analysis and inferences,

the data must meet tests of validity and reliability (Russ-Eft, 1980). Validity demonstrates

the degree to which the instrument actually measures that which it is intended to measure

(Russ-Eft, 1980). Furthermore, a survey instrument that is reliable can be expected to

achieve consistent results when used repeatedly.

Validity can be further broken down into two types: external validity, which

explains how well the data can be generalized over differing situations; and internal

65

validity, which answers the question of how well the instrument measure what it is

intended to measure (Cooper & Schindler, 2008). In order to ensure both the validity and

reliability of the research, a field test was conducted using a small group of individuals

who represented the target population in coordination with the research site agency. The

researcher screened the questions for vague and misleading language, as well as testing

and screening for vague terminology and poorly formed construct statements. Next, the

results were tested using factor analysis and Cronbach’s alpha to help ensure validity and

reliability of the survey instrument.

Ethical Considerations

Ethical considerations surrounding the use of any survey instrument that involves

the use of human subjects are not only an important part of ensuring external validity of

the research but also an essential part of ensuring that the research is conducted in a way

that protects these subjects from any harmful consequences. Important aspects of these

ethical considerations are ensuring that survey participants understand the voluntary

nature of the survey, are willing participants of the survey through the use of acceptable

informed consent practices, are assured of complete anonymity during the survey

process, and finally can be assured that no personal information will be collected which

could be used to identify them or attach them to the survey in ways that would be harmful

to the survey participants (Cooper & Schindler, 2008).

Before this survey was conducted, the questions were vetted. The primary ethical

considerations included the principles of informed consent of the individual and

reasonable protection of the individual from harm during any research involving human

participants (Fowler, 2002). The cover page of each survey contained information

66

regarding privacy protections and informed consent. Reasonable procedures were used in

this research to insure both of these principals were met. The survey did not ask for any

data that could possibly be used to identify the subject. A full disclaimer appeared on the

cover page of the survey questionnaire explained that participation in the research was

voluntary, along with the fact that there were no demographic data fields that could

reveal specific personal attributes of the participant.

The research site is the regulatory/oversight agency in public procurement, which

also owns and operates the national e-procurement system. One of the international

development banks has provided a grant to the government to strengthen the public

procurement reform through the development and implementation of the e-procurement

system. The international development bank for the project hired the researcher as an

international advisor on e-procurement. The researcher’s participation is limited in the

mission meetings with the research site officers, and also different meetings to review the

project progress. Because of this relationship, the researcher is familiar with some of the

e-procurement-project-related officials but there was no conflict of interest because the

researcher drew research data from an outside review of the project.

This research analyzes the implementation of the e-procurement system in the

research site country on dimensions of good governance by measuring the user’s

perception of the role of e-procurement, and their attitude towards its process and

operation. The professional relationship between the researcher and the participants does

not pose any bias, as the research analyzes outside perceptions of the new e-procurement

system by outsiders, not the participants that the researcher was advising.

67

There are three separate groups of respondents participating in the research, and

the researcher took clear measures to ensure there is no influence or bias in bringing them

into the research and evaluation. The survey administration tool used by the researcher

was designed to guarantee total anonymity of the survey participants. The survey and the

data collection tool were designed to hide any identifying features that could have

allowed the researcher to identify any specific personal identification of the survey

participants. The respondents participating in the research were treated with fairness,

respect and confidentiality. The professional relationship did not pose any risks to the

respondents, as the researcher did not posses any financial or other kinds of influence

upon the respondents. Data collected from the respondents are removed of all identifying

information. The confidentiality of the personal information, if any, of the participants

are separately maintained in an electronic file and each of the respondents are assigned a

code number. Only the code number was used to identify the related group, and for the

analysis of the responses. Respondents were all nonnative English speakers. English is

not a native language in the research site country but English is the second common

language. Translation services were not required for any of the respondents.

68

CHAPTER 4. RESULTS

Site Description

The government of the research site has initiated governance reform program in

various areas for effective public service delivery. Among others, public procurement is

one of the prime areas of reform of the government. As a part of this reform agenda, the

government has issued the Public Procurement Act and Public Procurement Regulations

to make the public procurement system transparent, fair, competitive and efficient. These

two legislations aim to maximize good governance through an effective and efficient

public procurement by equal competition among bidders, fairness, accountability and

reliability in public procurement processes. Also, as mandated by the act, public

procurement regulatory and monitoring agency was established. The overriding objective

of the agency is to implement the act effectively as well as to deliver efficiency and

value-for-money in the use of public funds adhering to the principles of public

procurement.

The public procurement regulatory and monitoring agency is implementing a

National Portal of e-GP System as a unitary platform for the procurements of goods,

works and services for all the public entities in line with the Public Procurement Act and

Public Procurement Regulations. The first phase of e-government procurement (e-GP)

system was designed to facilitate the bidder to prepare an annual procurement plan,

prepare bidding document, download bidding documents, prepare bids, and submit bids.

69

The second phase of e-GP system also has already been started with new functions and

incorporating the feedback and recommendations from the stakeholders based on the

experience of the first phase of implementation. The second phase also incorporated most

of the recommendations based on the findings from this action research. Use of e-GP

system is intended to increase transparency, nondiscrimination, equality of access, and

open competition. Pilot implementation of e-GP started in April 2013. The e-GP site also

provides easy to use Internet access for tender information, information on award of

contracts.

One of the international development banks is supporting the e-GP

implementation introduced under the Strengthening Public Management Program.

Research Methodology Applied to Data Collection and Analysis

The following research questions were asked:

1. Has the use of e-procurement system or e-government procurement system in public procurement enhanced the good governance in public procurement sector?

2. Does e-procurement add value to public procurement? If e-procurement increases (a) transparency; (b) accountability; (c) corruption control; (d) efficiency, effectiveness, and predictability; (e) easy access; (f) rule of law and equity; and (g) civil society awareness, then it is worth the investment.

With the increased trend of using e-procurement systems across the world by

government, proponents of the e-procurement, including Hidayanto et al. (2012), claim

that e-procurement plays a significant role in enhancing good governance facilitating

different aspects of good governance by the leverage of the Internet technology and

underlying best practice procurement processes, functions and its governance framework.

This study explored those claims.

70

Respondents were selected using purposive sampling to ensure a variety of public

entities. A questionnaire survey was used to gather data from at least two respondents

responsible for procurement using the e-procurement from each of the entities.

Respondents were managers, procurement professionals and also the ones who were

involved in the implementation of e-GP system in their respective public entities. The

names of the respondents have been kept anonymous due to confidentiality because this

study studies the role of e-procurement in good governance in the public procurement

sector in the country of the research site.

The survey questionnaire included 70 questions regarding seven constructs

considered to be the dimensions of good governance in public procurement, and

additionally there were four questions on a respondent’s profile in a view to get reliable

and valid data on the perceptions, attitudes, and general knowledge for each variable. For

the meaningful analysis of each of the variable, reliability and validity analysis was

carried out.

Analysis, Synthesis, and Findings

The research showed that the implementation of e-GP system is well accepted by

all the stakeholders despite having several obstacles. Again, a synthesis of several of

these obstacles indicates that the research site agency as an implementing agency appear

to have much more of a tactical than a strategic focus.

Other than a compilation of synthesis of analysis and findings with

recommendations based on dimensions of good governance as an outcome of this action

study in Appendix A, some other implementation issues must be analyzed here:

71

• Despite many scholar literatures recommend phased implementation of e-GP implementation as one of the essential requirement for the success of e-GP system, it appeared from the research that the phased approach of implementation e-procurement system without having implemented complete set of key processes is not always favorable for achieving desired target. The country started with the phased implementation approach starting with functions covering online bid submission, and then other phases. Within the bid submission phase also the government took phased approach of allowing hardcopy and softcopy submission at the beginning, and change to only online submission in later phases. If phase implementation is planned, the phase should be designed in such a way that it demonstrates the effectiveness of implementation and integrity of the process of procurements are maintained, not that it is phased to see if the system works or not, or functions implemented are complete or not. Purpose of the implementation should be clear at the design phase. Phasing is not effective if something is done electronically and other functions of the process are done in manual system. In absence of fit-for-purpose planning in phasing, actual benefit could not be achieved.

• Slow adoption of e-GP system is the impact of continuity of options accepting parallel submission of bids both in hardcopy as well as through e-procurement system for the same tender. Because both the options are open for the bidders, it is comfortable for them to stay with the status quo.

• Continuity of requirements to submit hardcopy original bid security and other documents, which brings bidders and procuring entities face-to-face, opening doors for potential corruption.

• Many procurement processes are not automated, which does not really reduce the administrative efforts but adds additional burden to procuring entities.

This research asserts that the implementation of e-GP system significantly

enhances the good governance in public procurement sector, however it suggests that a

comprehensive and practical e-GP implementation framework is absent in the body of

knowledge on e-procurement. This research also strongly establishes the theory that only

a synergetic and integrated approach of e-GP implementation can bring significant

improvement in good governance. If only one or two principles of procurement are

addressed by e-GP, then the rest of the principles should be addressed by traditional

72

approach, which creates huge disparity in delivering service. It rather reduces

performance, and hence outcome of reform agenda becomes insignificant.

In that sense, the outcomes of the study have impacted the overall perspective on

the e-GP implementation by the governments by highlighting and informing user of the

importance of focusing on the different aspects of good governance, rather than

minimally focusing on transparency, or efficiency, or saving costs and one or two

aspects.

Reliability Analysis

A commonly accepted rule of thumb is that an alpha of 0.6-0.7 indicates

acceptable reliability (although many state that this should be at least .7), and 0.8 or

higher indicates good reliability. Very high reliability (0.95 or higher) is not necessarily

desirable, as this indicates that the items may be entirely redundant. These are only

guidelines and the actual value of Cronbach’s alpha will depend on many things. For

example, as the number of items increases, alpha tends to increase too even without any

increase in internal consistency (Zaiontz, n.d.).

The ultimate purpose of designing a reliable instrument is to ensure that the scores

of similar aspects are relevant (i.e., internally consistent, contributing some unique

information to support the overall hypothesis).

As illustrated in Table 1, all dimensions of e-procurement good governance

factors have high levels of reliability with Cronbach’s alpha (α) ranging from 0.640 to

0.910. This ensures the reliability of the instrument designed for the survey. It suggests

that the questionnaire prepared for the survey are internally consistent, and contributes to

support the hypothesis. It confirms that the tests were designed with high quality and the

73

data received are reliable. The dimension Easy Access scored the lowest value (α =

0.640) and the dimension Accountability scored the highest value (α = 0.910).

Table 1. Reliability Analysis for Dimensions of Good Governance Factors

Dimension

Number of

applicablesurvey

questions

Sum of variance of all

questions (Σvar)

Variance of score totals

(var)

Cronbach’s alpha value

(α)

Transparency 24 27.442 115.114 0.795

Accountability 15 20.301 134.625 0.910

Corruption Control 4 5.470 13.972 0.811

Efficiency, Effectiveness, and Predictability 8 8.648 27.821 0.788

Easy Access 8 10.664 24.259 0.640

Rule of Law and Equity 7 3.814 9.385 0.692

Civil Society Awareness 4 4.717 11.182 0.771

Validity Analysis

Validity of the survey instrument is as important as reliability for the best

outcome from the study. Factor analysis was used to test the validity of the instrument by

identifying the interactions between the variables within the survey instrument. Although

final elimination of the questions can be a subjective process, factor analysis provides

important clues as to the validity of the questions in each construct (Cooper & Schindler,

2008).

There is no concrete formula for defining the adequacy of the samples, and one

can find many debates among academics about the adequate sample size to be used for

74

conducting factor analysis. Several early studies stressed the need for a minimum sample

number; however, later studies demonstrate that it is possible to conduct exploratory

factor analysis with small sample sizes (De Winter, Dodou, & Wieringa, 2009).

Additionally, Field (2000) showed that as long as the ratio of observations to variables is

at least 15, the results of exploratory factor analysis could be considered valid. Kaiser and

Rice (1974) recommended that Kaiser–Meyer–Olkin (KMO) measure should be

minimum 0.5 and values over 0.9 are excellent. In the survey study, 53 responses were

collected for each question; hence, the criterion of a minimum of 15 responses to each

question was met for the study. The KMO was used to measure the sampling adequacy

and Bartlett’s test of sphericity was used to test the null hypothesis that the variables in

the population correlation matrix were uncorrelated (García-Santillán, Chávez, Boggero-

Correa, & Vela-Aguilar, 2012). These two tests determine whether one can perform

further factor analysis on a particular construct (Walker & Madden, 2009). It is

concluded that the strength of the relationship among variables is strong. It is a good idea

to proceed with a factor analysis for the data.

In addition to the KMO measure and Bartlett’s test, other factor analysis results

considered for each construct were the total variance tables, the component matrices

tables, and the commonalities tables. For specific statistical results regarding validity and

reliability tests for each construct, please refer to Appendix B.

Validity and Reliability for the Constructs

Transparency. The KMO test for the Transparency construct had a value of

0.471, which is less than acceptable value of 0.5. The level of significance in Bartlett’s

test of sphericity result was only 0.000, which is not absolutely not significant. Based on

75

the argument of Foulger (1979), the researcher decided to accept the null hypothesis and

proceed for the further factor analysis. Additionally, the initial reliability test for the

transparency construct showed the significant internal reliability at the acceptable level of

Cronbach’s alpha = .795. Upon inspection of the questions, it appeared that when asked

together, the last question about the procurements adherence to the annual procurement

plan was redundant. As the research was more interested, in this particular construct, in

determining if the e-procurement system significantly contributes to enhance

transparency of procurement processes and information in public procurement, the

question could have been extracted from the list to enhance the Cronbach’s alpha and

improve the overall reliability.

For the Transparency construct, using the principal component analysis factor

extraction method and examining total variance explained, component matrix, and

communalities tables provided the basis for deciding that all the questions in this

construct could be correlated to the first eight factors (questions). The total variance

explained indicated that 73.405% of the total variance could be explained by the first

eight questions of the construct, as each of the eigenvalues for these questions are greater

than 1. Each one of these component questions remains in the survey. Additionally, the

component matrix demonstrated that all of the questions correlated well to at least one

other question in the construct. Finally, the communalities values showed the extracted

values for each question in the construct to be well over the required .45 level considered

minimally acceptable (Walker & Madden, 2009).

Accountability. The KMO measure and the Bartlett’s test for the Accountability

construct showed that further factor analysis was possible for this construct. The KMO

76

measure was .819 much above the minimum .5 required for sampling adequacy whereas

the significance level resulting .000 of the Bartlett’s test of sphericity showed the

accountability construct could be seen as a correlation matrix, indicating that further

factor analysis would be possible.

For the Accountability construct, using the principal component analysis factor

extraction method and examining total variance explained, component matrix, and

communalities tables provided the basis for deciding that all the questions in this

construct could be correlated to the first three factors (questions). The total variance

explained indicated that 70.046% of the total variance could be explained by the first

three questions of the construct, as each of the eigenvalues for these questions are greater

than 1. Each one of these component questions remained in the survey. Additionally, the

component matrix demonstrated that all of the questions correlated well to at least one

other question in the construct. Finally, the communalities values showed the extracted

values for each question in the construct to be well over the required .45 level considered

minimally acceptable (Walker & Madden, 2009).

The Accountability construct showed a highly acceptable level of reliability, with

a Cronbach’s alpha of 0.910. All the questions were fit for the final survey. No further

improvement of Cronbach’s alpha was required.

Corruption Control. The KMO measure and Bartlett’s test for the Corruption

Control construct showed that further factor analysis was possible for this construct. The

KMO measure was .748 above the minimum .5 required for sampling adequacy, whereas

the significance level resulting .000 of the Bartlett’s test of sphericity showed the

77

Corruption Control construct could be seen as a correlation matrix, indicating that further

factor analysis would be possible.

For the Corruption Control construct, using the principal component analysis

factor extraction method and examining total variance explained, component matrix, and

communalities tables provided the basis for deciding that all the questions in this

construct could be correlated to the first factor (question). The total variance explained

indicated that 65.783% of the total variance could be explained by the first question of

the construct, as the eigenvalue for the question is greater than 1. Each one of these

component questions remained in the survey. Additionally, the component matrix

demonstrated that all of the questions correlated well to at least one other question in the

construct.

Finally, the communalities values showed the extracted values for each question

in the construct were well over the required level of .45 except for the first question. The

extracted value for the first question was .429 marginally below the required level of .45.

Communality shows how much of the variance in the variables has been accounted for by

the extracted factors. For instance, 82.8% of the variance in response to the survey

question about the list of debarred/suspended bidders being publicly listed and

automatically verifying the winning bidder against the debarred/suspended bidders’

database is accounted for, whereas 42.9% of the variance in response to the survey

question about the e-procurement-system-based bidding requiring

suppliers/contractors/procurement personnel to submit or testify that they have not been

involved in corrupt activities in the past is accounted for.

78

The Corruption Control construct showed an acceptable level of reliability, with a

Cronbach’s alpha of 0.811. All the questions were fit for the final survey. No further

improvement of Cronbach’s alpha was required.

Efficiency, Effectiveness, and Predictability. The KMO measure and Bartlett’s

test for the Efficiency, Effectiveness, and Predictability construct showed that further

factor analysis was possible for this construct. The KMO measure was .828 much above

the minimum .5 required for sampling adequacy and the significance level of Bartlett’s

test of sphericity being .000 showed the Efficiency, Effectiveness, and Predictability

construct could be seen as a correlation matrix, indicating that further factor analysis

would be possible.

Using the principal component analysis factor extraction method and examining

the total variance explained, component matrix, and communalities tables for the

Efficiency, Effectiveness, and Predictability construct, it was determined that all the

questions in this construct could be correlated to the first two factors (questions) in this

construct. The total variance explained for this construct showed that 58.898% of the

total variance could be explained by the first two questions of the construct, with these

eigenvalues all being greater than 1. These two components remained in the survey.

Additionally, the component matrix showed all of the questions correlated well to at least

one other question in the construct.

Although the commonalities values showed the extracted values for each question

in the construct to be well over the required .45 level considered minimally acceptable

(Walker & Madden, 2009), there were two questions that did not correlate to the

extracted factors as well as the other questions in the Efficiency, Effectiveness, and

79

Predictability construct. The extracted measure for the question about the enhancing of

efficiency and effectiveness in public procurement process was .484 and the extracted

measure for the question about ensuring the value-for-money was .404. Because the

questions were important for assessing the perception of the users and the overall validity

and reliability of the construct were acceptable, the questions remained in the survey

questionnaire.

The initial reliability statistics for the initial two survey questions about the

availability of evaluation and contract management modules and adequacy of authority

given to public entity users indicated an unacceptably low level of reliability of .788 for

the entire construct. However, the reliability of the entire Efficiency, Effectiveness, and

Predictability construct could have been improved considerably if those two questions

were removed or elaborated more. When the entire construct was looked at holistically,

the question about value-for-money could have been misunderstood, as thus so far, the

value-for-money in procurements is not taken so seriously.

Easy Access. The KMO measure and Bartlett’s test for the Easy Access construct

showed that further factor analysis was possible for this construct. The KMO measure

was .633 above the minimum .5 required for sampling adequacy and the significance

level of Bartlett’s test of sphericity showed .000 and the Easy Access construct could be

seen as a correlation matrix, indicating that further factor analysis would be possible.

Using the principal component analysis factor extraction method and examining

the total variance explained, component matrix, and commonalities tables for the Rule of

Law and Equity construct, it was determined that all the questions in this construct could

be correlated to the first three factors (questions) in this construct. The total variance

80

explained for this construct showed that 63.853% of the total variance could be explained

by the first three questions of the construct, with these eigenvalues all being greater than

1. These three components remained in the survey. Additionally, the component matrix

showed all of the questions correlated well to at least one other question in the construct.

Although the communalities values showed the extracted values for each question

in the construct to be well over the required .45 level considered minimally acceptable

(Walker & Madden, 2009), there were two questions that did not perfectly correlate to the

extracted factors as well as the other questions in the Easy Access construct. The

extracted measure for the question about the bid document reuse was .597 and the

extracted measure for the question about bid preparation using online forms was .588.

Although both of these measures were above the generally accepted value of .45, they

were both significantly below the extracted scores of all of the other questions.

Upon further examination of these two questions in the group discussions, it was

identified that neither of these questions were available in the e-procurement system so

far. The questions were not removed from the survey, as these questions were important

for assessing the perception on the international best practices.

The initial reliability statistics for the Easy Access construct indicated an

acceptable level of reliability of .640 for the entire construct. However, the reliability of

the entire Easy Access construct could be improved considerably if the questions about

bid document reuse and bid preparation using online forms were removed.

Rule of Law and Equity. The KMO measure and Bartlett’s test for the Rule of

Law and Equity construct showed that further factor analysis was possible for this

construct. The KMO measure was .637 above the minimum .5 required for sampling

81

adequacy and the significance level of Bartlett’s test of sphericity showed .000 and the

Rule of Law and Equity construct could be seen as a correlation matrix, indicating that

further factor analysis would be possible.

Using the principal component analysis factor extraction method and examining

the total variance explained, component matrix, and commonalities tables for the Rule of

Law and Equity construct, it was determined that all the questions in this construct could

be correlated to the first two factors (questions) in this construct. The total variance

explained for this construct showed that 54.473% of the total variance could be explained

by the first two questions of the construct, with these eigenvalues all being greater than 1.

These two components remained in the survey. Additionally, the component matrix

showed all of the questions correlated well to at least one other question in the construct.

Although the communalities values showed the extracted values for each question

in the construct to be well over the required .45 level considered minimally acceptable

(Walker & Madden, 2009), there were two questions below the needed level. The

extracted measure for Question 4 about enhancing the citizen trust was .409, and

Question 7 about ensuring equal opportunity for all bidders—domestic or international,

small or large, and from cities or remote villages—was .364. Although both of these

measures were above the generally accepted value of .45, they were both significantly

below the extracted scores of all of the other questions.

Upon further examination of these two questions in the group discussions, it was

identified that neither of these questions were easy to answer. In case of enhancing trust

by e-procurement, all of three groups of stakeholders believe positively but it is too early

to correctly respond to this question. In case of Question 7, although procuring entities

82

and regulatory/oversight agencies believed the technology could provide such

opportunity, it was confusing in the bidding community because of the heterogeneous

distribution of the Internet connectivity and access to information and communications

technologies could be the barrier for the country. The questions were not removed from

the survey, as these questions were important for assessing the perception on citizen trust

and the equal treatment to the users through the use of e-procurement system.

The initial reliability statistics for the Rule of Law and Equity construct indicated

an acceptable level of reliability of .692 for the entire construct. However, the reliability

of the entire Rule of Law and Equity construct could be improved considerably if the

questions about citizen trust and the equal treatment to the bidders were clarified upfront.

Civil Society Awareness. The KMO measure and Bartlett’s test for the Civil

Society Awareness construct showed that further factor analysis was possible for this

construct. The KMO measure was .752 above the minimum .5 required for sampling

adequacy and the significance level of Bartlett’s test of sphericity showed .000. The Civil

Society Awareness construct could be seen as a correlation matrix, indicating that further

factor analysis would be possible.

Using the principal component analysis factor extraction method and examining

the total variance explained, component matrix, and commonalities tables for the Civil

Society Awareness construct, it was determined that all the questions in this construct

could be correlated to the first factor (question) in this construct. The total variance

explained for this construct showed that 66.682% of the total variance could be explained

by the first question alone of the construct, with these eigenvalues all being greater than

83

1. The component remained in the survey. Additionally, the component matrix showed

all of the questions correlated well to at least one other question in the construct.

Although the communalities values showed the extracted values for each question

in the construct to be well over the required .45 level considered minimally acceptable

(Walker & Madden, 2009), the first question was significantly below the needed level.

The extracted measure for the first question about the adequacy of the information

published by the procuring entities was .007. Upon further examination of the first

question in the group discussions, it was identified that the procuring entities were seen

reluctant to disclose adequate information for the general public as the procuring entities

are still not sure how much information to disclose to the general public. There is still no

policy and information classification system in the country for such information

disclosure. The question was not removed from the survey, as the question was important

for assessing the perception on the information disclosure on public procurement by

public entities to the civil society through the use of e-procurement system.

The initial reliability statistics for the Civil Society Awareness construct indicated

an acceptable level of reliability of .771 for the entire construct. However, the reliability

of the entire Civil Society Awareness construct could be improved considerably if the

question about information disclosure was clarified upfront.

The good governance construct results are summarized in Table 2.

84

Table 2. Summary of Good Governance Construct Results

Construct Kaiser–Meyer–Olkin measure

Bartlett’s test of sphericity

Cronbach’s alpha value (α)

Transparency 0.471 0.000 0.795

Accountability 0.819 0.000 0.910

Corruption Control 0.748 0.000 0.811

Efficiency, Effectiveness, and Predictability 0.828 0.000 0.788

Easy Access 0.633 0.000 0.640

Rule of Law and Equity 0.637 0.000 0.692

Civil Society Awareness 0.752 0.000 0.771

Data Collection

The survey questionnaire was designed, tested and finalized initially up to Step 8

of the utilization-focused evaluation (Patton, 2012a). After revision of the questionnaire,

67 participants were recruited from public entities, bidders, and regulatory/oversight

agencies. Among them, 35 responses were received from the public entity users, 11

responses from bidders, and 6 responses from the regulatory/oversight agency users were

received within 12 weeks. Fifteen participants did not return the responses. The

hypothesis testing that the e-procurement system contributes to enhance good governance

in public procurement was accomplished through the responses of 52 participants.

Analysis: Demographics of Survey Participants

The questionnaire was sent to a total of 67 participants. In total 52 respondents

from the three groups of respondent, public entities, bidders, and regulatory/oversight

agencies completed the questionnaire. The respondent profile is presented in Table 3.

85

Table 3. Study Respondent Demographics

Demographic Respondent %

Age group (in years) Under 25 1 1.92% 25–45 27 51.92% 45–65 24 46.15% Over 65 0 0.00%

E-procurement system user type Regulatory/oversight agency 6 11.54% Public entity 35 67.31% Bidder 11 21.15%

Years of e-procurement use Less than 2 years 23 44.23% 2–5 years 22 42.31% More than 5 years 5 9.62% Not applicable 2 3.85%

Annual participation in e-procurement Less than 5 times 5 9.62% Between 5–10 times 12 23.08% Between 10–20 times 11 21.15% More than 20 times 24 46.15%

Over 66% of the 52 respondents were from 14 public entities, 22.64% of the

respondents were from private firms, and two regulatory/oversight agencies using e-

procurement system.

Over 9% of the respondents were using e-procurement system for more than 5

years either in their own organizational level or using the national e-procurement system,

over 43% of the respondents have been using e-procurement system for the last 2–5

years, and over 43% of the respondents are new using for less than 2 years. Two

respondents from the oversight agencies were concerned with the e-procurement system,

but they do not use the system for their oversight purposes as of now, as the e-

procurement system still does not provide such access for the oversight agencies.

86

Over 47% of the respondents participate in the e-procurement for bidding

purposes more than 20 times, 20.75% participate between 10–20 times, 22.64%

participate between 5–10 times, and only 9.43% participate less than 5 times.

Among the participants, 52.83% were between 25–45 years old, 45.28% were

between 45–65 years old, and only one respondent was less than 25 years old. No one

from the participants was over 65 years old.

In summary, all the stakeholder groups were represented in the survey and focus

group discussions. Contacting public entities was easier as the entities had designated

focal persons to contact for the coordination, but contacting bidding community was

comparatively difficult because of their engagements in different contract in different

parts of the country.

Analysis: Hypothesis Testing Results

This analysis formulates inferences using the constructs of the seven specific

variables of the study. The hypothesis testing was done using both the Friedman test and

the chi-square goodness-of-fit test for nonparametric samples. This test compares three or

more related variables without making assumptions about their distribution. Each

question in a variable construct was itself considered a separate variable, with the entire

construct being tested in its totality using the Friedman test. For each of the tested

variables—Transparency; Accountability; Corruption Control; Efficiency, Effectiveness,

and Predictability; Easy Access; Rule of Law Equity; and Civil Society Awareness—if

there were no effect by any of the questions on each variable, the Likert rankings would

be spread out at random over the variables. However, if the null hypothesis is rejected,

then it is assumed that at least one of the questions invoked a significant opinion about

87

the effectiveness of the variable in question (Laerd Statistics, n.d.). The composite null

hypothesis was that the e-procurement system does not significantly contribute to

enhance good governance in public procurement. As the hypothesis test shows Friedman

test (χ2 = 139.623, df = 6, sig. = .000) at the α = 0.05 level of significance, there is

enough evidence to conclude that the e-procurement system significantly contributes to

enhance good governance in the public procurement sector.

As the study used a discrete ordinal, 5-point Likert scale with unknown

distributions of any of the variables and could not assume normality of the responses, chi-

square tests (Friedman and goodness-of-fit tests) were used either to reject or fail to reject

the null hypothesis for each variable. Discussion of the results for each of these variables

and their constructs follows.

Data analysis tables for each variable and corresponding construct can be found in

Appendix C.

Survey Results

Transparency

The null hypothesis for this variable was defined, as the e-procurement system

does not significantly enhance transparency of procurement processes and information in

public procurement. Twenty-four questions were asked concerning the transparency in e-

procurement that contributes good governance in the public procurement sector. The

study covered the questions to assess the perception and experience of the respondents in

the e-procurement system on the publishing of the Public Procurement Act and Public

Procurement Regulations, e-procurement guidelines, publication of annual procurement

plans, publication of bidding opportunities, disclosure of condition of participation,

88

evaluation criteria, criteria on domestic preferences, publication of amendments and

clarification to bidding document, sharing bid opening records, publication of contract

award details, level of automation in e-procurement system, and other aspects of the best

practices in e-procurement.

The Friedman chi-square test for the goodness-for-fit confirmed that e-

procurement contributes to enhance transparency in public procurement, as the test

rejected the null hypothesis on all the 24 questions at the α = 0.05 level of significance.

Most of the questions reflected the respondents’ perceptions that the e-procurement

significantly contributes to transparency aspect of good governance.

Minor exceptions in the transparency construct have to do with the features about

the transaction traceability through audit trails (Question 16), and ensuring the

availability of budget before publishing invitation to bid in e-procurement system

(Question 23). The chi-square tests for these questions had the significance level at the

higher range (i.e., significance level was .033). The questions asked about the

transparency in e-procurement were designed to understand the perception of the

respondents on the degree of importance of each of the transparency features in e-

procurement, and availability of those features in the e-procurement system the

respondents are using.

The data shows that the respondents understand that the existing e-procurement

system does not provide adequate traceability of the procurement processes and also

lacks the functionality to ensure the availability of budget before publishing invitation to

bid. These functions should be made available in the second phase of the e-GP system

implementation.

89

Accountability

The null hypothesis for this variable was defined as the e-procurement system

does not significantly contribute to enhance accountability of the actions in public

procurement. Fifteen questions were asked concerning the accountability in e-

procurement that contributes good governance in the public procurement sector. The

study covered the questions to assess the perception and experience of the respondents in

the e-procurement system on the provisions of the keeping records of approvals process

and approvers at different stages of procurement. In procurements, the records of

prequalification and postqualification of bidders, negotiation with them, details of

payments made and contract amendments are very crucial for the judgment of

accountability of the procurement officials. Equally important is the responsiveness of the

authorized public officers on timely handling complaints submitted by the bidders and

other decision making processes before contract awards and during the contract execution

time.

It was found that the respondents believe that the e-procurement strengthens the

accountability of the public officers through standard workflow process based on

incorporated authority in the e-procurement system. The Friedman test (χ2 = 195.092, df

= 6, sig. = .000) and chi-square test for the goodness-for-fit confirmed that the e-

procurement contributes to enhance accountability in public procurement, as the test

rejected the null hypothesis on 14 out of 15 questions at the α = 0.05 level of significance.

Fourteen questions reflected the respondents’ perception and experience that the e-

procurement significantly contributes to accountability aspect of good governance.

90

One exception in the accountability construct is about the potential suppliers

having effective and timely access to complaint review decisions and those complaints

are promptly resolved (Question 11). The chi-square goodness-for-fit test for the question

had high level of significance = .078. Therefore, it is assumed the respondents did not

experience the effective implementation of the feature about complaint access, review

and resolution in the e-procurement system.

Corruption Control

The null hypothesis for this variable was defined as the e-procurement system

does not significantly contribute to enhance corruption control in public procurement.

Four questions were asked concerning the corruption control in e-procurement that

contributes good governance in the public procurement sector. The study covered the

questions to assess the perception and experience of the respondents in the e-procurement

system on the provisions bidding requiring the suppliers/contractors/procurement

personnel submitting or provision of testifying that they have not been involved in

corrupt activities in the past. The study also seeks procuring entities record feedback on

experience with suppliers at the completion of the contract to help public officials in

evaluating the supplier for future procurements. The e-procurement system triggers

system alert in case of officials try to use their discretion and unauthorized judgment,

which provides a level of measure for controlling corruption and achieving value-for-

money.

Analysis found that the respondents agree that the e-procurement system helps

some level of controlling corruption, and the system provides tools for such control. The

Friedman chi-square test for the goodness-for-fit confirmed that the e-procurement

91

contributes to enhance accountability in public procurement, as the test rejected the null

hypothesis of all the questions at the α = 0.05 level of significance. Fourteen questions

reflected the respondents’ perception and experience that the e-procurement significantly

contributes to accountability aspect of good governance.

Efficiency, Effectiveness, and Predictability

The null hypothesis for this variable was defined as the e-procurement system

does not significantly contribute to enhance efficiency, effectiveness, and predictability

of procurement processes and information in public procurement. Eight questions were

asked concerning the efficiency, effectiveness, and predictability in e-procurement that

contributes good governance in the public procurement sector. The study covered the

questions to assess the perception and experience of the respondents in the e-procurement

system on the improvement in terms of effectiveness and effectiveness in procurement

process:

• Reduction in processing time

• Ensured adherence to the annual procurement plan and timely procurement

• Reduction in the number of complaints and appeals against the decisions made by the public entities

• Provided procurement information and procurement plans upfront so that bidders get time to plan

• Improved administrative process (i.e., automated workflow with defined authority and roles)

• Enhanced predictability on decision timings

• Ensures value-for-money

92

The Friedman test (χ2 = 61.423, df = 7, sig. = .000) and chi-square test for the

goodness-for-fit confirmed that e-procurement contributes to enhance accountability in

public procurement, as the test rejected the null hypothesis of all the questions at the α =

0.05 level of significance. All eight questions reflected the respondents’ perception and

experience that the e-procurement significantly contributes to efficiency, effectiveness,

and predictability aspects of good governance.

Easy Access

The null hypothesis for this variable was defined as the e-procurement system

does not significantly contribute to enhance easy access to procurement processes and

information in public procurement. Eight questions were asked concerning the easy

access in e-procurement that contributes good governance in the public procurement

sector. The study covered the questions to assess the perception and experience of the

respondents in the e-procurement system in terms of browsing speed of e-procurement

system and Internet bandwidth consumption, time required for getting registered to use

the system, affordability of registration fee, availability of previewing the bidding dossier

free of cost before buying the document, reusability of bidding dossier and bid

documents, availability of online forms for bid offer preparation, and completeness of the

e-procurement system to complete above 70% of procurement cycle through the system

itself.

The Friedman test (χ2 = 99.792, df = 7, sig. = .000) and chi-square test for the

goodness-for-fit confirmed that the e-procurement contributes to enhance easy access to

procurement processes and information in public procurement, as the test rejected the

null hypothesis of all the questions at the α = 0.05 level of significance. Eight questions

93

reflected the respondents’ perception and experience that the e-procurement significantly

contributes to easy access aspect of good governance.

Rule of Law and Equity

The null hypothesis for this variable was defined as the e-procurement system

does not significantly contribute to enhance the enforcement of the rule of law and equity

to all the bidders in public procurement.

Seven questions were asked concerning the rule of law and equal treatment to the

bidders in e-procurement that contributes good governance in the public procurement

sector. The study covered the questions to assess the perception and experience of the

respondents in the e-procurement system that helps in terms of

• Complying with procurement laws and regulations

• Help in public procurement reform

• Standardizing the public procurement process for better administration across all the procuring entities in the country

• Enhancing citizen trust on public service connecting its stakeholders

• Improving the administrative process in public procurement

• Treating e-procurement system as an integral part of the public procurement reform

• Ensuring equal opportunity for all bidders—domestic or international, small or large, and from cities or remote villages

The Friedman test (χ2 = 35.992, df = 6, sig. = .000) and chi-square test for the

goodness-for-fit confirmed that the e-procurement contributes to enhance the

enforcement of the rule of law and providing equal treatment to all the bidders in public

94

procurement, as the test rejected the null hypothesis of all the questions at the α = 0.05

level of significance.

Seven questions reflected the respondents’ perception and experience that the e-

procurement significantly contributes to enhance the enforcement of rule of law and

equity aspects of good governance.

Civil Society Awareness

The null hypothesis for this variable was defined as the e-procurement system

does not significantly contribute to enhance the public procurement awareness about

public procurement. Four questions were asked concerning the provisions made in the e-

procurement system that contributes good governance in the public procurement sector.

The study covered the questions to assess the perception and experience of the

respondents in the e-procurement system asking whether the information disclosed by the

public entities are adequate, whether public online discussion forum on public

procurement is available, whether e-procurement system provides provision of

independent access for review for auditing to auditing agencies such as the Office of

Auditor General and oversight agencies such as the National Vigilance Centre, the Anti-

Corruption Bureau, Parliament, and ombudsmen/mediators, and whether e-procurement

system informs the civil society about the government expenditure that help in terms of

enhancing civil society awareness so that it can play a role of watchdog.

The Friedman test (χ2 = 41.699, df = 3, sig. = .000) and chi-square test for the

goodness-for-fit confirmed that e-procurement contributes to enhance the civil society

awareness in public procurement, as the test rejected the null hypothesis of all the

questions at the α = 0.05 level of significance. Four questions reflected the respondents’

95

perception and experience that the e-procurement significantly contributes to enhance the

civil society awareness aspects of good governance.

Two-Way Analysis of Variance Test

These data were then analyzed using measures of central tendency, which

included the most common measures of mode, median, and the mean (Bernard, 2000),

factor analysis. A two-way analysis of variance test (see Table 4) was conducted in order

to determine if the research questions and the underlying hypotheses stated were

supported or not supported as well as to see the significance of the interactions between

dimensions (constructs or factors). These tests were conducted on sample data collected

under with seven constructs supported by 70 nonparametric questions from three groups

of e-procurement users of public entities, bidders, and regulatory/oversight agencies

involved in both procurement transactions and e-procurement system management. Table

4 values are not individual data points; they are all cell means with their own distribution.

In a two-way analysis of variance test, the way the distributions fit together is at

the very heart of the technique. Looking at the average mean data for each of the

stakeholder groups, it can be concluded that the stakeholders have similar perception

about the underlying hypothesis. The perception on the role of e-GP system in enhancing

the enforcement of the rule of law and create equal opportunities to the bidders scored the

highest support, and increasing efficiency, effectiveness, and predictability scored the

second. The Corruption Control dimension scored the least score as the e-GP improved

corruption control at certain stage but there are still other stages at which there is room

for improvement. All the stakeholders perceive that the transparency and easy access to

procurement information and bidding opportunities are well taken care by e-GP system.

96

Table 4. Two-Way Analysis of Variance Test on Research Questions and Underlying Hypotheses

Mean

Dimension of good governance Public entity Regulatory/

oversight agency Bidder Dimension

Transparency 3.31 3.48 3.17 3.32

Accountability 2.74 2.79 2.82 2.78

Corruption Control 2.64 2.42 2.80 2.62

Efficiency, Effectiveness, and Predictability 3.30 3.85 3.78 3.64

Easy Access 3.00 3.21 3.58 3.26

Rule of Law and Equity 4.02 4.14 3.86 4.01

Civil Society Awareness 2.57 2.38 3.14 2.69

Average mean 3.08 3.18 3.31 3.19

Then, the marginal means values from Table 4 were plotted in a graph to see the

interactions between the dimensions (see Figure 6).

An interaction occurs when effect of one factor changes for different levels of the

other factor. In this case, the perception of stakeholder groups changes across the

dimensions of the good governance. For example, bidder perception was good in the case

of Transparency; Efficiency, Effectiveness, and Predictability; Easy Access; and Rule of

Law and Equity but it was not good for Accountability and Corruption Control. The

perception the Civil Society Awareness dimension was moderate. If there was no

interaction, the perception of the bidder line would always be above the other stakeholder

user group lines, and lines would not cross. On the marginal means graph, as a general

rule, lines cross because factors change across the groups.

97

Figure 6. Marginal means of dimensions of good governance.

It is observed from the graph (see Figure 6) that three of the groups share the

same perception that there are interactions between the transparency and accountability.

Public entities and oversight regulatory agencies perceive that there is a relationship

between corruption control and efficiency, effectiveness, and predictability. Bidders and

Regulatory/oversight agencies believe that easy access to the information have significant

role on efficiency, effectiveness and predictability. Three of the focus groups perceive

that the e-procurement system is highly effective to enhance the rule of law and

competition among the bidders providing equal access to all. Regulatory/oversight

98

agencies and bidders are more optimistic on that the e-procurement system enhances the

civil society awareness.

Focus Group Discussions

As a part of the research design, three focus group discussions were held with the

participants from the public entities, bidders, and regulatory/oversight agencies separately

after the preliminary analysis of the questionnaire survey, to validate and deepen the

understanding of the findings of the quantitative analysis, and establish the concluding

hypothesis of the study.

Focus groups are particularly useful when participants’ reasoning behind their

views is of interest, as well as the process by which participants develop and influence

each others’ ideas and opinions in the course of discussion. Guidelines provided in the

Steyaert and Lisoir (2005) indicated that the focus groups are useful to gauge the nature

and intensity of stakeholders’ concerns and values about the issues, obtain a snapshot of

public opinion when time constraints or finances do not allow a full review or survey

obtain input from individuals as well as interest groups, obtain detailed reaction and input

from a stakeholder or client group to preliminary proposals or options collect information

on the needs of stakeholders surrounding a particular issue or concept, and determine

what additional information or modification may be needed to develop consultation

issues or proposals further (p. 131).

A set of guiding key questions was prepared for the discussions. Opinion, issues,

feedback, concerns, and suggestions were noted in each of the discussions. Sequence of

discussion was agreed by the participant to follow question by question and by theme

(aspects) of good governance. Full consideration was taken to understand the levels of

99

knowledge or different attitudes that may be relevant to the research and reflect on the

expected generalizability to the population at large (Steyaert & Lisoir, 2005). During the

discussion, themes were categorized into two categories, Perceptions and

Recommendations, for each of the constructs. And the perceptions and recommendations

for different themes that emerged from discussion were lumped when a consensus was

made in the group. Because the focus group discussion was carried out in small groups,

interaction was intense, and productive. Opinion of the participants and findings

deepened the understanding of why the survey scores were like they were, and in some

cases the discussion made clear why some cases that showed the outliers. The number of

voices really did not matter.

A total of 19 people participated in the three focus groups, 10 from public

procuring entities, five bidders, and four from the regulatory/oversight agencies.

Participants candidly put forward their perceptions, experience and suggestions during

the focus group discussion.

Transparency

Public entities. The public entities concluded that the transparency does not mean

to disclose everything. There should be clear policy on the disclosure of the information

to support the principles of the procurement and enhance good governance in the public

procurement sector. The transparency policy for the e-procurement should be prepared

based on the country context following the international best practices.

The participants noted that making the annual procurement plan and publishing

contract award information mandatory in the e-procurement portal was a good decision,

but preparation and adherence to the plan could be very difficult. The difficulty arises in

100

matching the time of the preparation and approval of annual budget by the Ministry of

Finance, preparation of the annual procurement plan, and the publication of invitation to

bids. The annual procurement plan in the e-procurement portal gives prior knowledge to

the general public as well as the bidders about the planned procurements for the fiscal

year, and gives information on how and where the public fund is being spent. The bidders

will have opportunity to plan upfront their resources, capacity and management planning.

A few of the participants also expressed that an estimated budget also could be

published to reduce the problem of low bidding. The low bidding is the bid by the bidders

quoting very low just to win the contract. Issues like low quality of work, abandoning the

contract in the middle of the execution, and abusing the mobilization funds are rampant

because of low bidding. The participants suggested including the price index (schedule of

rate) of the common procurement items in the e-procurement system, which gives the

reference for the bidders, as well as public entities can take as a basis for preparing an

official estimate for the procurement.

In relation to the invitation to bids, the participants suggested that the e-

procurement system should not allow to public invitation to bids until the budget is

secured for that specific procurement. In the current version of the e-procurement system,

bid-opening records are not shared with the bidders and bidders do not get to know the

status of bids until contract is awarded. Also there is no online public discussion forum in

the e-procurement for asking about, sharing about, and clarifying the procurement-related

issues. The participants mentioned that there is no option for debriefing in e-procurement

system, but expressed that it would be good to have in the system so that the bidders

101

failed to win the contract will get to their weaknesses and improve in next bidding

process.

Bidders. The participants expressed that the public procurement has been more

transparency with the introduction of e-procurement system. They are happy that the

annual procurement plan and the invitation to bids by the public procuring entities using

e-procurement system. The participants think that the information on eligibility and

qualification criteria also should be published in the invitation to bids, and those criteria

should not be allowed to modify after publishing the invitation to bids. The participants

requested to publish the bid opening records as soon as the bids are opened. They think

that the information published on contract awards is not adequate. It does not tell on what

basis the contract was awarded or who the competitors were. They believe if those details

are published then the bidders could understand their merits and weaknesses and will be

able to prepare better bids next time. The participants suggested that the e-procurement

system should automatically send the notes of prebid conference, and clarifications to

bidding document to all the bidders timely. The participants also suggested publishing the

best evaluated bidders with their scores, and debriefing should be available.

Regulatory/oversight agencies. The participants expressed that the public

procurement has been more transparency with the introduction of e-procurement system

in the country complying with the regulatory framework and Rights to Information Act.

Publication of annual procurement plan was not mandatory so far in practice before the

introduction of e-procurement.

The public procurement regulatory and monitoring agency informed that the e-

procurement system is going to be interfaced with the treasury and budget planning

102

software of the Ministry of Finance, so that e-procurement system can easily verify the

availability of budget for the procurement, confirm the approved procurement plans, and

reconcile with the payments made to the contractors. The participants agreed that bid

opening records, and detailed contract award information should be published. The

participants mentioned that publishing the eligibility and qualifying criteria should be

possible and it should be considered in the next version of the e-procurement system. The

participants also agreed that based on the international best practices, debriefing of the

contract awards should be given if the bidder requests for that. The regulatory/oversight

agency participants agreed that the complaints and their review decisions should be made

transparent.

All the participants fully agreed that the e-procurement system has significantly

enhanced the transparency in public procurement. Additionally the participants provided

feedback to improve or provide new functionalities like debriefing, online public

discussion forums, publishing list of best evaluated bidders, and complaints and their

review decisions, which will ensure the transparency as well as good practice in public

procurement.

Accountability

Public entities. The participants felt that the accountability in the procurement

has been enhanced through the introduction of e-procurement system, as the system

keeps track of activities of the procurement officers from login to logout in the system.

They also agreed that there are provisions to be considered by the regulatory and

monitoring agency as asked in the questionnaire survey to include in the e-procurement

system. For example, e-procurement system does not ask for signing any code of conduct

103

by the procurement personnel involved in the procurement processes for nondisclosure or

confidentiality.

The participants agreed that the e-procurement system should give login access to

all the authorized and accountable officers of the procurement so that they can carry out

their responsibility by themselves. Currently, in some cases, user login facility is given to

only one person from a public entity, which may result into repudiation by the

responsible officer. In many public entities, procurement personnel are not trained in

using computer systems and applications. Accountability is diluted when the procurement

officers share user name and password to the person who knows computer or to the

information technology staff, who carry out the actual operations in the e-procurement

system on their behalf. Frequent transfer of employees is another problem in case of

accountability. Also the procurement officials often do not use their official e-mail

addresses, so the notifications go to their personal e-mail boxes. Therefore, when

procurement employees are transferred, the user account and procurement-related

information received through the e-procurement system are lost. The participants

suggested either using the e-mail account of the respective public entity or that the e-

procurement system should provide e-mail accounts for all of its users. As per the

procuring entities, only 5% of the government officials use official e-mail accounts.

The participants expressed that the e-procurement system still does not provide

contract management function, which is the major area in which accountability issues are

weak. Additionally, the participants strongly recommended that the use of official e-mail

should be made mandatory for the public officer using e-procurement system.

104

Bidders. The participants argue that the public entity officials are not adequately

given authority to make decisions. The participants suggested that the public entity

officials could be made more accountable by giving appropriate authority, defining

timeline for actions, rewards for accomplishments and punishment or room for

improvement should be provisioned, and those should be built in to the e-procurement

system. The workflow process is in the e-procurement system but should be linked with

their threshold of approving authority and given mandates or roles. In case of bidders’

accountability, it is good that all the activities of the bidders are also recorded by the e-

procurement system, which makes the bidders feel accountable to their actions and they

cannot repudiate their actions.

Regulatory/oversight agencies. The participants expressed their satisfaction on

that the e-procurement system is based on workflow, and all the activities of the public

officers and bidders, as well as the users of the regulatory and monitoring agency are

recorded. The accountability of the actions are enhanced as the transactions and persons

doing the transactions could be tracked and investigated if required. The participants

suggested that the role-based authority given by the government to the public officers for

the procurement should be built in to in the e-procurement system and also suggested to

incorporate robust contract management system in the e-procurement system.

The participants suggested that if the quality assurance, payments, contract

variations, role-based authority of officers defined, complaint and dispute managements

are incorporated in the e-procurement system, it will drastically enhance the

accountability and good governance in public procurement.

105

Corruption Control

Public entities. The participants agree that the corruption control in the

procurement has been significantly enhanced through the introduction of e-procurement

system. Corruption is a vague phenomenon, which is rampant in different stages of the

procurement life cycle. The participants fully agree that face-to-face interaction should be

completely stopped in e-procurement system to minimize the corruption in procurements.

All of the participants agree that a regular phenomenon of clash among the bidders on the

last day of bid submission preventing each other from dropping bid documents is

completely stopped because of the e-procurement. Internal control in the public entity on

procurement also has been increased because the e-procurement system records all the

activities. Currently, there are at least one to two moments when bidders have face-to-

face contact with the procuring entity even in case of using e-procurement system. The

first event is based on the provisions made in the existing Public Procurement

Regulations. Bidders are supposed to submit the original copy of bid security to the

procuring entity to ensure the commitment of the bidders in the specific tender. The

second event is when the procuring entity may request the bidder to submit the hardcopy

of the bid, which was submitted online through the e-procurement system, if the bidder is

among the significantly responsive bidders. At the time of hardcopy submission, public

entities come to know who has submitted bids online, and that information may be passed

to other bidders, and from there a collusive act may start. The participants suggested to

stop the requirement of parallel submission of bids online and hardcopy as well as to find

the ways to receive bid securities electronically.

106

Bidders. The participants expressed that the corruption at the prebid submission

stage of procurement has been controlled because of the online availability of bidding

documents, and online submission of the bids. The bidders admitted that in many cases

corruption from the time of bid security submission and contract signing to contract

completion are still not controlled. The participants suggested that there should be an

easy way to submit bid securities, and hardcopy submission of bids should be completely

stopped. They also admitted that there is no strict control mechanism for the quality

assurance, payments, expenditures, and site monitoring. The e-procurement system

should address these recommendations by automating all the activities including the

submission of bid securities, and all information required to complete a bidding process.

The e-procurement system can provide greater control by developing a comprehensive e-

contract management system, linking with the quality control, payments and expenditure

tracking mechanism in the Ministry of Finance. The participants expressed that there may

be resistance from the public entities side to implement the contract management system;

however, it will be very helpful for the contractors.

Regulatory/oversight agencies. The participants agreed that the process-based

corruption control is significantly enhanced but it is difficult to control the corrupt

activities happening outside the e-procurement system. In many cases, contractors also

abuse payments made for the mobilization of the contracts by not utilizing the received

fund for that contract but investing it somewhere else. The e-procurement system should

have effective tools to track and control the payment, fund expenditure by the contractor

and outcome of that, and linkage with the delivery milestones. The regulatory/oversight

agency participants asserted that the incidences of fights among bidders at the time of bid

107

submission are not recorded so far since the introduction of e-procurement system. The

participants also agree that e-procurement has reduced collusion among bidders and

procuring entities by providing information about tenders to firms, allowing noncartel

firms to participate and breaking up local bidding cartels. E-procurement can also

potentially mitigate corruption by reducing the degree to which government officials

selectively withhold information or refuse to take bids from nonfavored bidders.

The participants believe that when all the processes of procurement are automated

and incorporated in the e-procurement system, corruption control can be sustained and

can significantly enhance the good governance.

Effectiveness, Efficiency, and Predictability

Public entities. The first important factor for efficiency gain is that any bidder

can now access the government procurement opportunities from a single e-procurement

portal. Bidders can prepare their bids online and submit the bid at their own pace online

till the deadline of online submission time. The one clear finding here was that the

requirement of parallel submission both online bid and hardcopy bids should be

completely stopped, and allow only online submission. This has created a redundancy in

the public entities requiring additional resources to verify the information in the bids

submitted online and offline.

Standard bidding document templates for the procurement of goods and services

are available as web forms, so it is easier to prepare the bidding documents, and it has

drastically reduced the bid document preparation time. There is built-in workflow for the

approval process, so the time for administrative processing also has been reduced, and

predictability of the procurement and contract processes has been increased. The

108

participants believe that cost reduction in the unit rates of the items are observed but that

should be carefully calculated over the time. Sometimes, because of the low bidding, the

public entities are facing problem in contract execution time. The participants also

requested to link the bid submission and evaluation function with the database of

blacklisted or suspended contractors and suppliers for verification. The e-procurement

system should have provisions of generating ad hoc reports on specific projects,

expenses, bid securities and performance guarantees validity tracking, forfeiture, release

and extension, and so on. The participants believe that if the contractor appraisal is tied

with the contract closure, it will help enhance the effectiveness. The e-procurement

should not end its scope just after a contract is awarded. It was suggested to develop a

comprehensive project management tool to achieve the significant enhancement in

efficiency, effectiveness, and predictability.

The participants agree that the mandatory requirement of publishing annual

procurement plan, standardization of the bidding document and forms, availability of the

administrative workflow has enhanced the efficiency, effectiveness, and predictability in

procurement processes.

Bidders. The participants in the focus group discussion expressed that the bid

securities and performance guarantees submission should be made simple, so that the bid

submission process will be efficient for the bidders. Currently, the bid securities are

submitted in hardcopy within 3–5 days of bid opening. The participants agreed that the

efficiency and effectiveness has been enhanced because of online bid document

preparation and submission. Predictability is enhanced because of the availability of the

annual procurement plans.

109

Regulatory/oversight agencies. The participants believe that efficiency and

effectiveness is enhanced by saving cost and time through the reduction in publishing

invitation to bids in daily newspapers, savings on printing bidding documents by public

entities, and improvements in standardized workflow across the public entities.

Requirement of preparing annual procurement plan and publishing it for public made the

activities predictable to the public entities as well.

The participants from three of the groups believe that the efficiency,

effectiveness, and predictability in the procurement have been significantly enhanced

through the introduction of e-procurement system by standardization of the procurement

processes, bidding documents, forms, and provision of online bid submission. Cost

reductions is achieved through reduction in publishing invitation to bids in e-procurement

system than in daily newspapers, reduction in the use of papers, and by administrative

process simplification. Stopping the parallel submission of bids, online and offline, and

development of a comprehensive contract management tool may even significantly

support the sustainability of these aspects in the procurement processes.

Easy Access

Public entities. The participants from the public entities are happy that all the

documents related to one procurement are available online in one place and can be easily

accessed. Despite the slow Internet connectivity, bidding documents can be prepared

easily and in a short time. Although some of the participants requested to upgrade the

data center and also increase the Internet connectivity for the e-procurement, most of the

participants were happy with the current speed of access. They did not have any issues

related to user registration, as the registration of the public entities is done by the

110

regulatory and monitoring agency itself in the current version of the e-procurement

system. The participants suggested improving on the consistency of web pages in terms

of content identity, font, and access links to functionalities.

Bidders. The participants praised the provision of free registration of the bidders

and also availability of downloading bidding documents without any cost. The

participants requested to build facility to send Short Message Service alerts along with

the e-mails to the bidders from the e-procurement system when any new procurement

opportunity of interest to the specific category of bidders is published in the e-

procurement portal. There should be a provision to indicate the area of business or their

interested business areas at the time of registration in the e-procurement system. This will

be useful and effective for the bidders to enhance access to procurement information. The

Short Message Service alert will encourage the small and big contractors to participate in

the government procurements based on their capacity.

Regulatory/oversight agencies. The e-procurement system still does not provide

dedicated content access to the regulatory/oversight agencies. It was recommended to

give access to the regulatory/oversight agencies on the reports on compliance and

performance tracking, as well as audit trail records of the transactions on case-by-case

basis.

The participants believe that the easy access to the procurement processes and

information have been significantly enhanced through the introduction of e-procurement

system. The participants suggested to enable Short-Message-Service-based automated

alert to the bidders on the new bidding opportunities in the e-procurement system. The

participants also suggested improving on the consistency of web pages in terms of

111

content identity, font and access links to functionalities. It was recommended to provide

regulatory/oversight agencies access to the reports and also transactions audit trails for

the investigation of the specific cases.

Rule of Law and Equity

Public entities. The participants believe that the compliance with the Public

Procurement Act and Public Procurement Regulations, and also equal treatment to the

bidders have been significantly improved, as the e-procurement system is configured

according to the requirements of the act, regulations, and government directives,

provisions of which are automatically verified by the system at every stage of transaction.

Procurement workflows are configured based on the administrative authority and paper-

based standard bidding documents are reengineered and built in to in the e-procurement

system, ensuring the standardized procurement processes and documentations. Bid

submission has been easier as the system does not allow submitting the bids until the

bidders submit all the requested documents, which in turn support to increase the

responsiveness of the submitted bids. Bidders are treated equal, as the status of the

bidders remains anonymous to the procuring entity until the time of bid opening. The

participants suggested making the feature of automatic verification of contract approval

thresholds for different officers assigned for different roles in the workflow, so that the

chance of making mistake to approve any transaction beyond their authority could be

checked. It was discussed also on the issue of authenticity of the electronic transactions.

Despite having electronic transaction act in the country, the participants raised the

concerns that all the legal, technical and other provisions should be documented in the

form of e-procurement guidelines with the endorsement from the Ministry of Finance or

112

Cabinet of Ministers to give a legal status of the procurements carried out through the

electronic medium. This may enhance the citizen trust on e-procurement as well.

Bidders. The participants agreed that the compliance with the requirements of the

procurement has been very easy, as the e-procurement system does not allow to submit

the bids until the requirements are fulfilled. The e-procurement can now ensure the

compliance even for those who are not aware about the Public Procurement Act and

Public Procurement Regulations. The participants expressed that there should be easy

way to submit complaints in the e-procurement system. They also expressed that there

should be e-procurement manuals to follow the right rules in case of e-procurements.

Regulatory/oversight agencies. It has been easier for the regulatory/oversight

agencies also because of the streamlined process and standardization of documents in

compliance with the Public Procurement Act and Public Procurement Regulations. The

participants agreed that there should be e-procurement guidelines with adequate legal

status to ensure the authenticity of the practices and transactions carried out through the

e-procurement system.

The participants in the three of the focus groups believe that the rule of law and

equity in procurements have been significantly enhanced through the introduction of e-

procurement system through the incorporation of standardized procurement processes,

workflow with administrative authority, and automated compliance checking by the e-

procurement system. The participants recommended making public e-procurement

guidelines to give authenticity to the electronic procurement transactions. They also

recommended that the contract approval threshold of the official in the procuring entity

should also be available in the workflow process.

113

Civil Society Awareness

Public entities. The participants believe that the information published about

procurement plans, invitation to bids, contract award information in the e-procurement

system are adequate for the awareness of the general public. Currently, the regulatory and

monitoring agency has access to the administrative control panel and the agency is

monitoring the regulatory and fiduciary compliance in the e-procurement system. The

participants suggested providing such access to the Public Procurement Review

Committee as well, who are responsible for handling complaints and appeal procedures

of the procurement process. The participants believe that the civil society is now a lot

more positive, and encouraging the use of e-procurement in all the public procuring

entities.

Bidders. The participants believe that the level of general public awareness has

been raised about the e-procurement. There should be regular publication of annual

procurement performance reports in the e-procurement portal to inform the general public

about the government expenditure on the procurements.

Regulatory/oversight agencies. The regulatory/oversight agency participants

agreed that the complaints and their review decisions should be made public for the

public society awareness so that the civil society can play a role of watchdog in society.

The participants in three of the groups agree that the awareness about the public

procurement in general public has been significantly improved, because of the increased

transparency measures provisioned in the e-procurement system. The participants

recommended providing access on complaint handling issues to the Public Procurement

114

Review Committee, publish annual procurement performance report as well as publish

the decisions made on the complaints submitted by the bidders.

Summary

The action research examined seven aspects related to good governance in public

procurement within the framework of e-procurement system implemented in the host

country for this research. The data were analyzed using quantitative methods (survey)

and also qualitative methods (focus group discussions).

From the quantitative methods, for each of the variables in question the null

hypothesis was rejected with the alternative hypotheses being accepted. Three

participatory focus group discussions with the public procuring entities, bidders, and

regulatory/oversight agencies were carried out as the study was focused on values such as

transparency, accountability, merit, legality, and legitimacy, which, in the context of the

good governance paradigm require civic participation and engagement (Hood & Lodge,

2004). Three of the groups perceived and agreed that these aspects of good governance in

public procurement have been significantly enhanced because of the introduction of the

e-procurement in the country.

Action research also provided evidence on areas, which would be difficult to

identify otherwise. Within the constructs of each variable, specific questions provided

some very specific insights about good governance aspects through the e-procurement.

Results indicate that role of e-procurement in enhancing good governance in the

public procurement sector was positively significant.

115

A more detailed discussion of these insights, along with possible ramifications,

conclusions, limitations of the study, and opportunities for further research, is provided in

Chapter 5.

116

CHAPTER 5. DISCUSSION, IMPLICATIONS, AND RECOMMENDATIONS

This action research was focused on assessing the role of e-government

procurement (e-GP), or e-procurement, system in enhancing good governance in public

procurement sector. Specifically, the research explored user and stakeholder attitudes,

perceptions, and knowledge of key points in the e-procurement process in the country. It

has been seen that good governance has become a major concern of governments with a

focus on efficiency and value-for-money in expenditure in public procurements as well as

support for the sustainable socioeconomic development of any country. This research

supports the main premise that e-GP system does significantly support an enhancement of

good governance in public procurement.

The research was designed to discover how e-GP system users perceived and

defined the limitations of this support for the improvement of good governance.

Outcome recommendations have been developed that have already begun to

contribute in the behavior and attitudes of public organizations and other stakeholders.

This practical impact on the e-procurement system has occurred as a part of “a

collaborative process with the researcher and the intended users as they work within

focus groups to evaluate the program’s resources, gather data, evaluate findings, and

execute the recommendations” (Patton, 2012a, p. 154). The outcomes summarized in

Chapter 4 enabled the public procurement regulatory and monitoring agency and other

117

intended users to improve the implementation and use of e-procurement system “by

designing an intervention to create change” (Patton, 2012a, p. 57).

The outcomes of the study are expected to impact perspectives and use patterns in

e-GP implementation by governments around the world as the research findings highlight

and inform the importance of focusing on good governance as a goal, rather than

minimally focusing on transparency, or efficiency, or saving costs. The findings of the

evaluation allow the public procurement regulatory and monitoring agency to calibrate

the existing approach to enhance good governance and simultaneously provide a

comprehensive e-GP implementation framework for all countries beginning to implement

e-GP systems.

Revisiting the Problem

Presently, most of the governments around the world implement the e-

procurement systems as a tool for public procurement reform, but have a negligible

impact on significant change. Many countries are focusing merely on cost savings in the

use of e-procurement, or other benefits.

This action research analyzed the implementation of the e-procurement system in

the host country of this research in terms of its utility addressing different dimensions of

good governance by measuring the users’ perceptions about the role of the e-procurement

in public procurement good governance, and their attitudes towards its use. With this

approach, the research unveiled incomplete knowledge and practice of the underlying

attitudes, perceptions, and knowledge of significant roles of the e-procurement system

that could change and impact good governance.

118

Discussion of Findings, Implications, and Recommendations

Compilation of feedback and recommendations as an outcome of this action

research is attached in Appendix D and a set of functions and facilities recommended to

enhance good governance is provided in Appendix A. Findings, implications, and

recommendations for each of the constructs of good governances are discussed here in

details.

Transparency

The survey analysis rejected the null hypothesis that the e-procurement system

does not significantly contribute to enhance transparency of procurement processes and

information in public procurement. A further analysis into each question of the construct

for transparency revealed a very strong consensus about the significance of e-

procurement system contributing to good governance.

The e-procurement system publishes invitations to bid and procurement award

information but lacks few of the following important features, procedures, knowledge,

and transaction-related practices:

• Inadequate traceability of transaction—Respondents recognized that the existing e-procurement system does not provide adequate traceability of the procurement processes. In absence of traceability, the process and outcome of the process will not be transparent, and raises questions on fair treatment. The processes of the transactions should be made transparent to the bidders as well as other users of the system.

• No mandatory requirement to ensure availability of budget—The e-procurement system lacks the verification instrument to ensure the availability of budget before publishing invitation to bid. It points to the fact that the planning process is weak, and challenges the effective execution of the contracts. Any procurements should be planned upfront with adequate budget assurance from the Ministry of Finance. Automated verification with the financial management system in Ministry of Finance will address this problem.

119

• Absence of debriefing function—The e-procurement system does not provide debriefing of the reasons of the specific bidder not being selected for the contract. However, the bidders are notified about the winner of the contract through a manual communication. Debriefings should be provided to the bidders upon their request within stipulated time after the notice of contract awards.

• Absence of e-procurement guidelines—For the users of e-procurement system, it is not transparent how the whole e-procurement system works. Users lack adequate knowledge about how they are protected, and what are the consequences. The implication is that e-procurement system is running without a clear knowledge of legal and operational authenticity of e-GP, its underlying electronic processes and incorporated operational practices different from the traditional procurement approach, authenticity of the transactions done through e-GP by its users, ownership of data and content stored in the e-GP system, security and institutional arrangements for its governance, and roles, responsibility, and obligations of the users. The e-procurement is allowed by the Public Procurement Regulations, and data-security-related issues are addressed separately by the Electronic Transactions Act of the country, but a preparation of a comprehensive e-procurement directive will provide the clear operational governance framework of the e-procurement system. The e-GP guidelines is urgent in the context in which few agencies are carrying out their procurements through e-procurement system, and many public entities are still following the traditional paper-based procurement system.

• Trend of bidding low—An interesting phenomenon of low bidding was revealed in case of majority of bidding processes after the implementation of the e-procurement system in the country. In few cases the bids received were more than 50% less than the estimated price. Looking at this phenomenon only from the perspective of cost savings, it appears good. However, the respondents indicated that the phenomenon seems to be engendered because of perceived online competition. Another reason could be the reduced face-to-face contacts of the bidders with the public entity employees. The implication of this phenomenon is huge, which affects the socioeconomic development objective of the country. Low bidding raised a few challenges, such as (a) Contractors compromise on quality; (b) Contractors quit contract during its execution; (c) Contract remains incomplete; and (d) There is an increase in contract variations in terms of contract scope and cost. To resolve these challenges, a few options can be recommended: (a) Prepare price reference of the common procurement items by the public procurement regulatory and monitoring agency, and publish the updated reference price in the e-procurement system for the use by the public procuring entities and bidders. It will ensure the minimum quality standard, and also controls the price variations; (b) Disclose the estimated price of the procurement package in the

120

annual procurement plan; (c) Add function to controls bid price at the time of bid submission, if the bid is substantially low. Permissible minimum offer can be set in the system, but for this regulatory and monitoring agency should work out on the reliable price reference to be used for preparing the estimate of the procurements; (d) Increase the weight substantially in quality criteria, giving less weight to price in evaluating the bids; and (e) Introduce supplier appraisal and make it public. This approach makes bidders more responsible to their bids, and contract execution.

• Requires more stakeholder engagement—The survey and focus group discussions revealed that the respondents feel lacking of more stakeholder engagements. It shows that there is no platform online or offline to share the knowledge, clarify the issues, provide opinion, and there is no appropriate channel to contribute on the policy and strategy formulation on procurement-related issues. Based on the aspirations of the respondents of the research, it is recommended to make the e-procurement system social medial enabled by making available online discussion forums, introducing mobile Short Message Service alerts on procurement opportunities and other important events, and increase stakeholders engagements through the social media enablement (i.e., Facebook, Twitter, etc.) of the e-procurement portal by publishing bid opportunities, information on public procurement and encouraging open interaction on the public-procurement-related issues.

The public procurement regulatory and monitoring agency should review the

feedback and recommendations provided by the respondents through survey results and

focus group discussions as well as considering the international best practices in e-

procurement implementation, and implement in its subsequent updates and releases of the

e-procurement system implementation.

Although the stakeholder respondents fully agreed that the e-procurement system

has significantly enhanced the transparency in public procurement, further research and

policy level decisions are needed while incorporating and implementing these feedback

and recommendations. The public procurement regulatory and monitoring agency is

recommended to work with the e-procurement policy and implementation experts with

the engagement of all key stakeholders.

121

Accountability

The survey analysis rejected the null hypothesis that the e-procurement system

does not significantly contribute to enhance accountability of the actions in public

procurement. A further analysis into each question of the construct for accountability

revealed a very strong consensus about the significance of e-procurement system

contributing to good governance.

Although the null hypothesis on accountability was rejected and it was assumed

that there were perception, knowledge and adequate attitude towards the fundamental

requirements of the good governance in terms of accountability, examination into the

individual questions in this construct revealed that, in many cases, this knowledge was

incorrect and inadequate. Some of those gaps and challenges identified through the

research are discussed as follows:

• Absence of code of conduct—It was found that the e-procurement system does not bind the public officers, bidders, and other users by any kind of code of conduct, terms of use, or confidentiality and nondisclosure declarations. In many cases, surprisingly the perception on accountability revolved only around the public officers, whereas the same level of accountability is expected from all the actors (i.e., public officers, regulatory/oversight agencies, bidders and civil society) involved in the procurement process and outside for good governance in the public procurement sector.

As discussed in case of transparency, traceability was not possible in the e-procurement system. Although code of conduct is also not enforced, it is difficult to track the accountability of the actors in the process outside the e-procurement system. The accountability measures should be enforced both for the e-procurement system and outside the system. This kind of agreement to the code of conduct for the actors of procurement process in the e-procurement system also helps to maintain the confidentiality of the content and protects entities from the challenges of untimely disclosure of incomplete and unauthorized information.

• Diluted accountability—A shocking practice of sharing user ID and password by the authorized officers to the lower level staff or information technology operators to carry out transactions on their behalf, was revealed during the

122

focus group discussions, which puts the accountability aspect challenged if the challenge is not addressed. The challenge engendered by different factors: (a) The authorized officer is not confident using e-procurement system due to the lack of computer skills; (b) The administrative process is based on command-control model, in which the high-level officers do not execute themselves but give instructions to the lower level staffs; (c) The management of public procuring entities perceive that the use e-procurement is the function of information technology professionals and not the mainstream function of procurement management; and (d) The change management process and business process reengineering were not at all carried out or not adequately carried out.

It is recommended that a comprehensive change management plan with clear objectives, target group, activities and messages to be conveyed should be prepared, and capacitate the users in the new environment updating their perception, knowledge, attitude, and leveraging the potential of e-procurement in a real sense.

• Use of personal e-mails in official transactions—Another malpractice was revealed that the users using their personal e-mail accounts, obtained from free online e-mail services (i.e., e-mail accounts from Gmail, Yahoo, Microsoft, and other known and unknown service providers) in procurement transactions and communications in e-procurement system. The e-procurement system sends automated e-mails and transaction-related information to the e-mail account submitted at the time of their registration. The implication of this practices results into the loss of procurement communications, and leakage of confidential procurement information from these account. When employees are transferred to another organization or discontinue their employment, or have bad relationships with the authority of the organization, the information and communication carried out through the use of personal e-mails are lost and knowledge and accountability not retained in the organization. It is not traceable. The public procurement regulatory and monitoring agency is highly recommended to enforce an e-mail policy of using only an official e-mail account either organization domain name based e-mail account issued by the corresponding organization or, the account issued by the government for its employees, or the e-mail account offered by the e-procurement system. This will help to recover the information, and also reuse the same e-mail account by new employee just changing user password.

• Inadequate authority to procurement officers—One of the reasons of ignorance in accountability was revealed as the inadequate authority given to the public procurement officers for making decisions on varied and complex procurements. The implication of this challenge is the lengthened procurement cycle, misprocurement, misappropriation of the authority to execute contract, and capacity to manage the contracts. Public officers could be made more accountable only by giving appropriate and adequate authority, defining

123

timeline for actions, thresholds of their authority, rewards for accomplishments and punishment or room for improvement, which could be built in to the e-procurement system. A basic workflow process is there in the e-procurement system, but it should be linked with the threshold given for the approving authority, and linked with given mandates or roles.

• Contract management module—Currently, there is no contract management module in the e-procurement system, which leaves out the vast scope of accountability in terms of delivery of the contracts. All responsible actors should be accountable for ensuring the quality of deliveries within the agreed time and cost with the highest level of ethical behavior and attitude in executing the contracts. For this, a comprehensive contract management module with the functionalities and tools to track deliveries, milestones, payments, quality assessment, and appraisal of suppliers and contractors should be incorporated in the e-procurement system.

It appeared that many of the attributes associated with accountability were

attributed to public procurement entities and officers. What was surprising to the

researcher was that these perceptions were held across the spectrum of all three groups of

users who responded and participated to this survey. This lack of understanding about the

nature and role of accountability, especially the role played by accountable and decision

making authorities, may well have negative implications in shaping the perception,

knowledge, behavior and attitude of the entire organization (either it is public

procurement entity, or bidding firm, or regulatory/oversight agency) in e-procurement

environment with operational goals of the agencies. Further research should ascertain

whether the observed perceptions and attitudes of the respondents and participants were

actually based on their experience or are due to a lack of clear understanding about

accountability or a result of the transitional change process into an electronic

environment.

Although the respondents and participants indicated that they fully agreed the e-

procurement system has significantly enhanced the accountability in public procurement,

124

and they recognize important attributes of accountability, the perception, knowledge and

attitude towards the scope and spread of the accountability was somehow incomplete, so

further research is necessary to strengthen the nature and role of actors in the

accountability process.

Corruption Control

The survey analysis rejected the null hypothesis that the e-procurement system

does not significantly contribute to enhance corruption control of the actions in public

procurement. A further analysis into each question of the construct for corruption control

revealed a very strong affirmation about the significance of e-procurement system

contributing to good governance.

Although the null hypothesis on corruption control was rejected, and it was

assumed that there were perception, knowledge and adequate attitude towards the

fundamental requirements of the good governance in terms of corruption control,

examination into the individual questions in this construct and focus group discussions

revealed that, in many cases, the corrupt attitude was still rampant, and there are many

areas in which the e-procurement system should be upgraded incorporating measures for

corruption control. Some of the key findings and their implications and recommendations

are discussed as follows:

125

• Psychological impact on corruption control—According to the experience of the respondents and participants, corruption usually happening before contract awards have been significantly reduced. All of the participants agree that a regular phenomenon of clash among the bidders on the last day of bid submission preventing each other from dropping bid documents is completely stopped because of the e-procurement. The impact was the result of the online availability of bidding documents, and online submission of the bids, but in many cases corruption from the time of bid security submission and contract signing to contract completion are still not controlled. Anonymity of bidders before bid submission has affected psychologically to the bidders, so the coercive practices at that stage are completely stopped.

• Many key processes are not automated—Face-to-face interactions of bidders with the public officers, one of the fundamental sources of corruption, are still not fully stopped. To cite few of the examples, (a) Bidders are still required to submit original paper copy of bid securities to public procuring entities, whereas they submit their bids online; (b) In many cases, public procuring entities are still asking to submit a paper version as well even after submitting bids online; and (c) At the time of prebid opening, bidders are still invited to be present at the public procuring entity. In the matured e-procurement systems in other countries, online interactive question-and-answer sessions, and subsequent sharing of clarifications to queries on bidding documents already replace prebid meetings. In most of the matured e-procurement systems, bid openings are done by the e-procurement system, and the bid opening records are automatically shared with the participating bidders. It is evident that business process reengineering at the planning and design phase of the e-procurement system was not adequate. Many of those paper-based processes and practices could have been eliminated by other online options to avoid face-to-face interactions. The implication is that the bidders’ anonymity are disclosed, and the users from both the sides are still obliged to use manual as well as electronic processes. It does not really address the key requirements of corruption control. It is recommended to implement alternative options for avoiding face-to-face interactions. Direct interface with the banking network or providing access of securities handling module of e-procurement system to the commercial banks for bid-securities-related transactions could be an effective option.

• Major corruption in contract execution—The respondents and participants indicated that easy corruption is happening at the time of contract execution. There is no control and effective monitoring of the contract execution in the field. Participants, especially the bidders indicated that giving a certain percentage of contract payments as commission to the client is already established as a legal system in the civil works contracts. Major corruption is practiced also under the cover of scope change and contract variations. The e-procurement system can provide greater control by developing a

126

comprehensive e-contract management system, linking with the deliverables, quality control, payments and expenditure tracking mechanisms.

• Access to audit trails of procurement transactions—Oversight agencies do not have access to the audit trails of the transactions in the e-procurement system. Controlled access to such audit trails and other information strengthens the forensic investigations in case of any suspected fraud or corruption cases in public procurement.

The positive side of the e-procurement system is that it has enhanced the internal

control of the activities of the procuring entities in some degree by automatically

recording all the activities of the users carried out through the e-procurement system.

Although the respondents and participants indicated that they fully agree the e-

procurement system has significantly enhanced the corruption control in public

procurement, and they recognize that corruption is happening beyond the e-procurement

system as the system does not have control over the activities outside. Ethical aspects of

the procurement process can only be addressed through self-declaration by both the

public officers and bidders of their noninvolvement in any kind of fraud and corruptive

activities in and outside the e-procurement system. Innovative ways of developing

functions for corruption control could be explored and incorporated in the future versions

of the e-procurement system.

Efficiency, Effectiveness, and Predictability

The survey analysis rejected the null hypothesis that the e-procurement system

does not significantly contribute to enhance efficiency, effectiveness, and predictability

of procurement processes and information in public procurement. A further analysis into

each question of the construct for efficiency, effectiveness, and predictability revealed a

127

very strong consensus about the significance of e-procurement system contributing to

good governance.

The null hypothesis for the Efficiency, Effectiveness, and Predictability variable

was rejected. It appeared that there were extremely positive responses towards this

construct of good governance. Perception of all the three groups confirmed that the e-

procurement was contributing to enhance the efficiency, effectiveness, and predictability

in public procurement. However, an investigation into the questions, knowledge on

measures required in e-procurement system for enhancing efficiency, effectiveness, and

predictability aspects, and the attitude of the users revealed that the perception,

knowledge, and attitude were less than comprehensive. Some of the findings and their

implications and recommendations are discussed as follows:

• Phasing approach of e-GP implementation—The research reveals a significant flaw on the theory of phasing of e-procurement system implementation. The phasing of the system implementation was significantly fragmented. It started with the phasing of annual procurement plan preparation to e-bid submission, and second phase was planned to have bid opening, evaluation, contract awards, and contract management with payment functions. The first phase itself suffered from many gaps between dots. Many of the subprocesses of procurement process those included in the first phase had to be handled manually. That made the online processes also significantly inefficient. It appeared from the research that the phased approach of implementation e-procurement system without having implemented complete set of key processes is not always favorable for achieving desired target. If phase implementation is planned, the phase should be designed in such a way that it demonstrates the effectiveness of implementation and integrity of the process of procurements are maintained, not that it is phased to see if the system works or not, or functions implemented are complete or not. Purpose of the implementation should be clear at the design phase. Phasing is not effective if something is done electronically and other functions of the process are done in manual system. In absence of fit-for-purpose planning in phasing, actual benefit cannot be achieved.

• Absence of evaluation and contract management module—Because the e-procurement system still did not have evaluation and contract management

128

modules, as well as limited functionalities in workflow, efficiency, effectiveness, and predictability were not achieved in true sense. However, even with the limited facilities in the e-procurement system, the level of satisfaction received by the users on these aspects gives incentives to the stakeholders to further enhance the e-procurement system to achieve more. A well structured evaluation module for works, goods, and services contracts reduces the discretionary decisions on contract awards, standardizes the evaluation process, provides a level of predictability on the results to the bidders, and builds trust on e-procurement system.

• Lack of performance measurement and monitoring tool—The current version of the e-procurement system provides limited compliance validation to the Public Procurement Act and Public Procurement Regulations but it does not provide any indicators for the performance measurements. However, the system provides few standard reports.

• A standard bidding document template, annual procurement plan, and workflows are key. Availability of few standard bidding document templates, annual procurement plan and workflow have significantly enhanced efficiency, effectiveness, and predictability. If the workflow feature is linked with the roles and thresholds of authority for administrative and financial approvals, the system will be more effective.

• Lack of system knowledge—Some of the features are still not used because of lack of knowledge. One of such good features available is the facility for the public procuring entities to prepare the bidding document using the already used bidding documents, which could minimize the time for bidding document preparation. Another feature is that the procuring entities can prepare the bidding documents and schedule date for publishing invitation to bids, not requiring to wait till the scheduled time in the annual procurement plan to prepare the different sections of the bidding document (i.e., specifications, evaluation criteria, other supplementary details).

129

• Training requirement—Efficiency and effectiveness if in some way related to skills and capacity of its users. Because the e-procurement is new to the bureaucracy in the procurement sector, it requires extensive training on using the e-procurement system.

It could be that this lack of trainings on the details of the e-procurement system has permitted many misperceptions about efficiency, effectiveness, and predictability in the use of e-GP system. This seems to be an area that would benefit from follow-on research to determine if a link exists between the lack of training on the details of e-procurement system features and the somewhat limited knowledge displayed about aspects of good governance. However, based on the evidence revealed by this research, this seems to be a plausible reason.

Easy Access to Procurement Information and instruments

The null hypothesis for the Easy Access variable was also rejected that the e-

procurement system does not significantly contribute to enhance easy access to

procurement information, processes and instruments in public procurement. A further

analysis into each question of the construct for easy access revealed a very strong

consensus about the significance of e-procurement system contributing to good

governance.

Despite the null hypothesis for the Easy Access variable was rejected, the survey

respondents pointed out there were several barriers to e-procurement system

implementations. Although the respondents felt that e-procurement system

implementation was important for the country, they identified several obstacles that they

felt were preventing smooth implementation. Listed as follows are some of the findings,

implications, and recommendations:

• Adequate Internet speed—Speed of the Internet is one of such key obstacles. Although 85% of geographical areas of the host country of this research, including remote parts of the country, have access to Internet and 3G networks, consistency of connection and speed are challenging. The participants mentioned that the cost of the Internet is getting cheaper. Implication of the slow Internet speed may sometime creates problems of

130

incomplete bid submission or worse is when bid submission deadline is crossed. Most of the small bidders go to cyber-cafes to submit bids and even sometimes receive help from the public procuring entity to upload the bids. The later practice taking help of public procuring entity employees to submit bids conflicts with all principles of procurement creating conflict of interest. The observed practices in some cases, providing support to bidders on the bid preparation and submission by public entities should be stopped. To tackle the issue of Internet, it is recommended to develop an offline tool to prepare the bidding documents and bids without requiring Internet, and allow upload or submission when Internet is available. The regulatory/oversight agency, accepted that the Internet bandwidth in the data center at the Government Integrated Data Center was also as low as 5 mbps, and it is being upgraded in Phase II to 15 mbps along with the additional server and network equipment for failover arrangements.

• Access to bidding documents—The e-procurement system has taken a policy to provide free access to the bidding documents before submitting bids, which provides the bidders to access their own capacity, resources, confirm the eligibility and qualifying criteria, and other compliance requirements before investing time and resources in preparation of bids.

• Easy planning—The respondents and participants highly praised the policy to publish an annual procurement plan, which again helps both the bidders and public procuring entities to plan their resources upfront. Disclosure of estimated cost of the procurement and government reference price on common procurement items would help better planning for both the bidders and public entities.

• Lack of knowledge on complaint handling—It was revealed that the knowledge of bidders on their rights to make complaints on the procurement process and decisions was very weak. The e-procurement system does not provide any information on this knowledge and also there is no e-procurement provision to facilitate the rights of the bidders for lodging complaints. Complaint submission functions should be available for the bidders and should be easy to use.

Rule of Law and Promote Equal Treatment to the Bidders

The null hypothesis for the Rule of Law and Equity variable was also rejected that

the e-procurement system does not significantly contribute to enhance rule of law and

equity in public procurement sector. A further analysis into each question of the construct

131

Rule of Law and Equity revealed a very strong consensus about the significance of e-

procurement system contributing to good governance.

It was observed that the respondents and participants perceived the most

significant role of e-GP was to ensure the rule of law and promote the equal treatment to

the bidders. Some of the findings, implications, and recommendations are discussed as

follows:

• Issue of small bidders—Some of the participants from the bidder community expressed that there may be an unequal treatment to the small bidders, who do not have adequate access to information and communications technologies. However, not all bidders believe the same. There is a strong theory from bidders as well as other focus groups that even the small bidders can quickly make themselves ready in information and communications technologies if the bidders get an opportunity to bid on the government procurements and treat them equally with the other bidders.

• E-GP guidelines for enforcing rule of law—In absence of the e-GP guidelines, there are numerous ambiguities in case of enforcing the procurement acts and regulations in case of e-procurement system. The process and practices in the manual and electronic environments differ a lot, and those differences should be clearly communicated to the stakeholders.

• Mass communication campaign for bidders’ registration—Bidders registration and its adoption seems a bit slow. A critical mass of bidders should be registered in the system, so that a real competition can be achieved. If the number of bidders is not significant in the e-procurement system, it will have negative consequences in terms of equal treatment to the bidding community. It was also revealed that because of the limited number of bidders participating in the e-procurement system, award of the contracts sometimes appeared to be rotating only among a known group of contractors. The attitude of the bidders observed in this case is not favorable for the e-procurement system in the long run, however it provides incentive to the bidders at the initial period. A divide in the bidding community with access to e-procurement system and without will imbalance the procurement sector. It is recommended to start a campaign for the registration in the e-procurement system, and also mobilize the professional membership associations of suppliers and contractors to encourage and instill the knowledge about the e-procurement system. Only a critical mass of bidders registered in the e-procurement system, and competing in the procurements may bring value-for-money as well as promote the principle of equity in the procurements. In

132

absence of adequate number of bidders in the e-procurement system, it will only institutionalize the organized syndication, cartel, and other malpractices.

• Automatic compliance to the procurement rules—Only a limited rules are configured in the e-procurement system. Configuration of e-procurement processes based on the prevalent acts, regulations. and procurement rules facilitates the compliance of the procurement activities significantly. The e-procurement system helps to enforce the micromanagement steps throughout the procurement process. It is recommended a dedicated commitment to the enforcement of procurement rules by the regulatory/oversight agencies, and e-GP facilitates to enforce that. Compliance to the Public Procurement Acts and Regulations provides a legal basis for procurement contracts.

• Complaint handling—The e-procurement system does not provide function of complaint handling. Also, the role of the Procurement Review Board is not incorporated. The whole process of complaint submission and processing is done in manual environment. It discourages the bidders to submit the complaints even many genuine cases. It is recommended to incorporate the function for the Procurement Review Board and also provide easy access to such function to the bidders from their private dashboard.

Civil Society Awareness

The null hypothesis for the Civil Society Awareness variable was also rejected

that the e-procurement system does not significantly contribute to enhance civil society

awareness in public procurement sector. A further analysis into each question of the

construct for civil society awareness revealed a very strong consensus about the

significance of e-procurement system contributing to good governance.

First, the respondents, especially the oversight agencies mentioned that the tools

required for the civil society awareness is not adequate in the current version of the e-

procurement system, which cannot foster an environment that was conducive to engage

general public and civil societies as watchdog in the public procurements carried out

through the e-procurement system. Some of the findings, implications, and

recommendations are discussed as follows:

133

• Absence of citizen engagement—There was a strong urge from the bidding community to start an online public discussion forum for the discussions on the procurement-related issues and sharing knowledge among the stakeholders. Currently, there is no platform for expressing opinion about the procurement-related issues, and share the practices. If online discussion forum is started, it will also help shaping future policy and strategies for the enhancement of e-procurement system and procurement sector as a whole by informed and participative support from the stakeholders.

• Absence of procurement statistics—There is lack of public procurement performance statistic and analysis of the past procurements, which could be used as feedback for the future procurements. Currently, it is very difficult to find real statistics on the public procurement. General public, the taxpayers are not aware about the expenses of public fund. National plans on socioeconomic development and strategic procurements are made without reliable statistics of data, and hence the national level strategy and objectives do not align with the procurements made by the public entities. It is recommended that e-procurement system collect, analyze, and disseminate such analytical statistics and provides feedback to the national development strategy.

• Establishment of helpdesk—It is felt a requirement of setting up a helpdesk, which can bridge the civil society with the procurement sector. Currently, it still does not provide the helpdesk to ask questions in relation to procurements.

• Open data and open contracting—The country has started recently an initiative on open contracts through a support from the World Bank. It has opened up avenues for the disclosure of procurement- and contract-related public information, and makes the civil society aware about the public procurements and contracts, however there are no clear government policies towards open data and open contracting practices to date. It is recommended to join the global initiative on open data and open contracting to enhance transparency and accountability with information dissemination to the general public.

Despite a lack of many options for engaging civil society in public procurement,

the respondents and participants appreciated the existing level of information disclosure,

which were not at all accessible in traditional method of procurements.

134

The research concludes that the awareness about the public procurement in

general public has been significantly improved, because of the increased transparency

measures provisioned in the e-procurement system.

Implementation and Incorporation of Action Research Outcomes

The advantage of action research following utilization-focused evaluation is the

realization of outcomes into real-life actions during the evaluation process (Patton,

2012a). The outcome of the project and the utilization-focused evaluations enabled the

public procurement regulatory and monitoring agency and other intended users to

improve the implementation and use of e-procurement system “by designing an

intervention to create change” (Patton, 2012a, p. 57). According to Ramirez and

Broadhead (2013), “[utilization-focused evaluation] evaluators facilitate a learning

process with attention to how real people in the real world might apply evaluation

findings and experiences. In designing a utilization-focused evaluation, the attention is

constantly on the intended use by intended users” (p. 1). As the action is the only way to

generate and test new social science knowledge, the evaluation findings in participation

of intended users were prepared as the classified recommendations for use. The public

procurement regulatory and monitoring agency has prioritized the research evaluation

recommendations under Strategic and Operational categories, so that the use of findings

becomes more manageable. Most of the recommendations have been implemented in the

rollout version of the second phase of e-procurement implementations in the country. The

following recommendations have already been incorporated or are in the process of being

incorporated in the second phase of the e-procurement system:

135

• The agency has already initiated a process to amend the Public Procurement Regulations to allow only electronic submission of bids in case of procurements through the e-procurement system. It will stop the inconsistencies seen because of online- and paper-based hardcopy parallel bid submission on the first phase of the e-procurement system implementation.

• E-mail notifications to bidders notifying important events, including user registration, new invitation to bid publication, and notice of contract awards.

• Enhanced functionalities to reuse documents from the already completed similar procurements.

• Availability of functionalities to submit bids in part, and allow the bidders to modify and update the bids until final submission.

• Availability of bid opening, sharing of bid opening records with participating bidders, and evaluation modules.

• Availability of contract award notice, and debriefing on request, and publication of details on the awarded contracts.

• Availability of comprehensive contract management module with functions to track deliverables, milestones, quality assurance assessments, e-invoicing, and payments.

• Available solution for collecting payments for the bidding document purchase, issuance and management of bid securities and guarantees through banking channel.

• Inclusion of e-catalog with classification of goods, works, and services using one of the popular international standard for classification common procurement vocabulary.

• Standardization of bill of quantity formats for goods works and services.

• Introduction of more standard bidding document templates for works, goods, and services.

• Initiation to develop e-GP guidelines.

• Initiation for business process reengineering on the regulatory and monitoring agency, and Procurement Review Board processes on complaint handling and administrative review.

136

• Encryption of all transactional and other confidential data in database.

• Publication of debarment list of bidders.

• Mandatory requirement to publish annual procurement plan and ensuring budget for the procurement, and other functions.

These are the examples of how the action research recommendations contributed

to change in the behavior and attitudes of the public organizations and stakeholders at the

practical level as well, through “a collaborative process with the researcher and the

intended users as they work within focus groups to evaluate the program’s resources,

gather data, evaluate findings, and execute the recommendations” (Patton, 2012a, p. 154).

Conclusions

This research was intended to assess perception, attitude, and knowledge of the e-

GP users on the role of the e-procurement system in enhancing different dimensions of

good governance in the public procurement sector as well as the adequacy of measures

taken in the design and implementation of the e-GP system based on the best practices in

different countries. This action research appears to show that public procurement sector

stakeholders do consider e-GP plays very significant role in enhancing good governance

in the public procurement sector. The outcome of the study reveals a change in behavior

and attitude towards the e-GP system implementation by addressing gaps in different

aspects of governance e-GP implementation to meet the fundamental requirements of

good governance. Their research evaluation reveals that the e-procurement system can

enable good governance. In addition to the perception assessment, the action research

questions in this research were intended to assess the opinion of the participants

137

regarding the adequacy of measures taken in the design and implementation of e-GP

systems based on the best practices in different countries.

The findings of the evaluation provided input for the public procurement

regulatory and monitoring agency to allow calibrating the existing approach of

implementing e-GP system as well as provide a comprehensive e-GP system

implementation framework for the countries in the process of implementing e-GP

systems for enhancing good governance in public procurement.

The bidding community appreciates e-procurement when it reduces complex data

entry procedure, validates the data as it is entered by improving data quality, standardizes

and streamlines their work, makes the procurement processes transparent, treats equally

to all bidders, and provides provision of debriefing the reason of failure to win the

contracts, and the access to e-procurement system is hassle free. This action research also

revealed that there is considerable need to communicate the utilization of e-GP system to

enhance the good governance in public procurement to the stakeholders. As indicated by

the United Nations Educational, Scientific and Cultural Organization (2005), the e-

awareness amongst the government employees and their willingness to embrace change

shall play a key role in the whole process.

However, an evaluation of specific questions asked in the areas of transparency,

accountability, corruption control, efficiency, effectiveness, and predictability, easy

access, rule of law and equity, and civil society awareness led to the conclusion that there

are already many necessary aspects for good governance in public procurement sector

exist and can be incorporated within the e-procurement system. When viewed from the

experience from many other countries for the introduction of e-procurement systems, the

138

system implemented in the first phase in the country is still at an immature level and

inconsistently implemented, but provides important lessons to the world.

Nevertheless, based on responses about the role of e-procurement system in

enhancing different seven aspects of the good governance, further research may show

that functions, tools, processes and practices in terms of aspects of good governance

could well be improved. It is also obvious that the resultant data on perception, attitude

and knowledge could be different if the e-procurement system implementation is assessed

after few years.

Although there were significant levels of perception, attitude, and knowledge

about role of e-procurement systems to enhance different aspects of good governance,

respondents to the survey questionnaire and participants in the focus group discussions

exhibited some confusion about the specific roles and functions as well as pointed several

deficiencies in the existing version of e-procurement system. More specifically, the

research showed that the strategic nature of good governance is often confused with the

more tactical and operational aspects of e-procurement system management. It was

observed that all key aspects of good governance were not taken as a strategic-level

agenda for the public procurement through the e-procurement system, yet these aspects

orchestrated together only may impact as the key tenets of good governance in public

procurement sector.

It was quite intentional that questions about all aspects of good governance were

asked in the same questionnaire. Although detailed correlation of the variables is beyond

the scope of this current research, it appears that for the user of the e-GP system, there is

139

a real lack of opportunities for hands-on training, and awareness campaigns about the

aspects of good governance in the e-procurement system.

At the very least, it would appear that the stakeholders are optimistic about the

achievements so far from the e-procurement system, and look forward to its

improvements in the implementation of the next phase.

The research showed that there appeared to be several obstacles to implementing

e-procurement system. Again, a synthesis of several of these obstacles indicates that the

stakeholders appear to have much more of a tactical than a strategic focus. Despite many

scholar literatures recommend phased implementation of e-GP implementation as one of

the essential requirement for the success of e-GP system, it appeared from the research

that the phased approach of implementation e-procurement system without having

implemented complete set of key processes is not always favorable for achieving desired

target. If phase implementation is planned, the phase should be designed in such a way

that it demonstrates the effectiveness of implementation and integrity of the process of

procurements are maintained, not that it is phased to see if the system works or not, or

functions implemented are complete or not. Purpose of the implementation should be

clear at the design phase. Phasing is not effective if something is done electronically and

other functions of the process are done in manual system. In absence of fit-for-purpose

planning in phasing, actual benefit could not be achieved. One of the major challenges in

leveraging full potential of e-procurement system in the country is its continuity of

options accepting parallel submission of bids both in hardcopy as well as through e-

procurement system for the same tender. Another challenge is the continuity of

requirements to submit hardcopy original bid security and other documents, which brings

140

bidders and procuring entities face-to-face, opening doors for potential corruption.

Another challenge is that the many procurement processes are still manual, which does

not really reduce the administrative efforts, but adds additional burden to procuring

entities.

This research asserts that the implementation of e-GP system significantly

enhance the good governance in public procurement sector, however it suggests that a

comprehensive and practical e-GP implementation framework is absent in the body of

knowledge on e-procurement. A comprehensive and practical e-GP implementation

framework has been designed and proposed (Appendix E) based on the research, best

practices around the world, and experience of the researcher in assisting governments

around the world in introducing and implementing e-GP systems. It is believed that the e-

GP implementation framework will provide a practical guidance to the countries

considering implementing e-procurement as a mean for public procurement reforms.

E-procurement implementation should take into consideration of integrated

approach to the good governance. Mere focus on one or two aspects of the good

governance does not bring significant changes in the public procurement sector, there are

always one or another aspects lagging behind and the sector does not achieve desired

objectives. The e-procurement system technology acceptance and adoption theory has

been widely studied in developed countries. However, it has not been studied in the site

country of this research. Thus, this study is unique to the country context. Further, the

study can be served as a starting point for other e-GP or e-procurement adoption

researches in other countries. Moreover, the study provides the public procurement

regulatory and monitoring agency of the country with the type of empirical analysis,

141

which is necessary for fostering adoption and usage of e-procurement system to enhance

the good governance in public procurement.

At the beginning of this research, the synthesized definition of e-government

procurement used for this dissertation was that e-procurement is operationally defined as

the collaborative utilization of information and communications technologies (especially

the Internet) by government agencies and other actors in procurement community in

conducting all activities of government procurement process cycle for the acquisition of

goods, works, and services enhancing good governance in public procurement (Shakya,

2012).

An e-GP system is improvised and defined as the collaborative use of information

and communications technologies by government agencies, bidding community,

regulatory, oversight agencies, other supporting service providers, and civil society in

conducting ethical procurement activities of the government procurement process cycle

for the procurement of goods, works and service, and management of contracts ensuring

good governance and value-for-money in public procurement, and contributing to the

socioeconomic development of country.

Limitations of the Study

The scope of the study does not cover the final two steps, 16 and 17, of the

utilization-focused evaluation, although Step 16 was assessed partially to confirm

whether the findings and recommendations of the study has been considered for the

improvement. As Step 16 should cover the follow-up with users to use and Step 17

should reassure the evaluation through the consequent meta-evaluation of the evaluation

itself, substantial interval of time is required to carry out the meaningful accomplishment

142

of these steps. Thorough evaluation of Steps 16 and 17 are intentionally left out for the

future study.

Although the overall findings of this research prove the significant role of e-GP in

enhancing good governance in public procurement sector, it has some limitations worth

mentioning. Although the initial target population of the study was intended to be more

bidders, the project running in different places of the country made difficult to get access

to them and getting responses to survey questions from all of the expected bidders.

Although the public procurement regulatory and monitoring agency in the host country of

this research is in preparation of starting the second phase of the e-GP system

implementation, another limitation is that the e-GP system currently running in the

country supports only limited functions of the complete government public procurement

cycle, so the responses to many questions related to the functions beyond bid opening are

fully based on the perception and knowledge of the bidders at the current stage.

Nevertheless, although the researcher believes the perceptions and knowledge of bidders

are sufficient to make this research valid, the absence of few of the functions of

procurement cycle is an acknowledged limitation that might have introduced a bias in the

results. The diverse geographical location of the surveyed public procuring entities may

also limit any wider application of this research.

As pointed out by the participants in this research, a great majority of the

interventions in the implementation of the e-GP system are tactical in nature rather than

strategically focused. Additionally, these public procurement entities face unique

challenges created by easy availability of Internet access and electricity that other

developed countries may not face.

143

The method of data collection and the ethical considerations were taken care to

avoid the bias and avoid the limitation of the validity of the results. Additionally,

although this type of research has been done only for the few of the dimensions of the

good governance in public procurement, and never done research on the role of e-GP on

enhancing the comprehensive aspects of the good governance, the research has been able

to ascertain that this has been the first survey ever accomplished assessing the role of the

e-GP system to enhance good governance in the public procurement sector. As such, in

order to ensure that the survey results are externally valid, this survey would have to be

administered even to like populations in other countries.

Recommendations for Further Research

This study sought out preliminary perceptions, attitudes, and knowledge about the

role of e-GP in enhancing good governance in the public procurement sector. It provided

many initial insights about the perceived state of good governance, role of e-GP, and the

e-GP implementation by the users actively working within the public procuring entities,

bidders, and regulatory/oversight agencies, insights, which when augmented by more

detailed research, may provide information that could lead to improvements in e-GP

system implementation in the host country of this research and perhaps in other countries

as well.

Although this information provided clues as to the nature and state of good

governance and role of e-GP in enhancing good governance in public procurement sector,

this research could be enhanced further if the public procurement regulatory and

monitoring agency of the research site country or the specific country government or the

international organizations and development partners like the World Bank, Asian

144

Development Bank, European Union, United Nations, or multilateral development

organizations would consider sponsoring similar research.

This initial research showed that e-GP significantly enhances good governance

providing a wide range of functions to support different aspects of good governance but

that some confusion remained over the nature, scope, and role of e-GP system and

developing sound governance policies to support the sustainability of the e-GP system as

the mainstream procurement platform. This appeared to have its origin in the lack of

training and learning opportunities about e-GP and scope of good governance. If

governments want to positively impact long-term strategic good governance in public

procurement sector through the implementation of e-GP system, they should consider

further research on how best to more formally integrate training and education into every

phase of an employee’s career, engage citizens in learning process, and provide learning

opportunities to the bidding communities.

Finally, in order to effectively integrate e-GP system for enhancing good

governance into the administrative processes of the whole procurement cycle, further

research should be undertaken in the area of obstacles to implementation and change

management. This exploratory research indicated not only lack of functions and

interventions in implementation but also, more problematically, a lack of any

comprehensive e-GP implementation framework for the countries to follow. A

comprehensive e-GP implementation framework is designed based on the lessons learned

from the e-GP implementation experiences gathered around the world, and proposed as a

guidance to follow for the implementation of e-GP system for good governance in the

public procurement sector, making implementation effective (see Appendix E). The

145

researcher believes that the e-GP implementation framework could prove to be of great

value to enhancing effective implementation of e-GP system to enhance good governance

in a real sense in the public procurement sector.

Summary

In terms of e-procurement implementation, it should take into consideration of

integrated approach to the good governance. Mere focus on one or two aspects of the

good governance does not bring significant changes in the public procurement sector,

there is always areas lagging behind and the sector does not achieve desired objectives.

The e-procurement system technology acceptance and adoption theory has been

widely studied in developed countries. However, and it has not been studied in the host

country of this research. Thus, this study is unique to the country context.

Lack of appropriate government policy on e-Procurement, adequate business

process reengineering of existing procurement processes, scoping of complete e-GP

system, not having comprehensive e-GP guidelines for the smooth implementation of the

e-GP system, fragmented efforts of implementations by individual public entities their

own e-bidding systems, and still having online- and hard-copy-based parallel submission

of bids are challenges in the successful implementation and adaption of e-GP system

enhancing good governance in the public procurement sector.

Further, the study and the proposed e-GP implementation framework can be

served as a starting point for other e-GP or e-procurement adoption researches in other

countries. Moreover, the study provides the public procurement regulatory and

monitoring agency of the research site country with the type of empirical analysis, which

is necessary for fostering adoption and usage of e-procurement system to enhance the

146

good governance in public procurement. Outcome of this research should be used to

further develop and tailor existing capabilities of the e-GP system as well as the

capability of the agency that assist the e-GP users and other stakeholders of the public

procurement sector as they transition from legacy to electronic environment.

147

REFERENCES

Agarwal, R., & Prasad, J. (1997). The role of innovation characteristics and perceived voluntariness in the acceptance of information technologies. Decision Sciences, 28(3), 557–582. doi:10.1111/j.1540-5915.1997.tb01322.x

Al Moalla, A., & Li, D. (2010). Organizational issues with electronic government procurement: A case study of the UAE. The Electronic Journal on Information in Developing Countries, 41(3), 1–18. Retrieved from http://www.ejisdc.org/ojs2/ index.php/ejisdc/article/download/669/316

Ampratwum, E. F. (2008). The fight against corruption and its implications for development in developing and transition economies. Journal of Money Laundering Control, 11(1), 76–87. doi:10.1108/13685200810844514

Ararat, M., & Ugur, M. (2003). Corporate governance in Turkey: An overview and some policy recommendations. Corporate Governance, 3(1), 58–75. doi:10.1108/ 14720700310459863

Arrowsmith, S., Treumer, S., Fejø, J., & Jiang, L. (n.d.). Public procurement regulation: An introduction. Retrieved from http://www.nottingham.ac.uk/pprg/ documentsarchive/asialinkmaterials/publicprocurementregulationintroduction.pdf

Asia-Pacific Development Cooperation. (1999). Non-binding principles on government Procurement. Paris, France: The Chair of the Government Procurement Experts Group, Asia-Pacific Economic Cooperation.

Asian Development Bank, Inter-American Development Bank, & World Bank. (2004). Strategic electronic government procurement: Strategic planning guide. Retrieved from http://www.unpcdc.org/media/2622/strategic%20electronic%20 government%20procurement%20-%20strategic%20planning%20guide.pdf

Asiimbwe, B. (2012, November 22). Unclear policies hinder e-procurement. New Vision. Retrieved from http://www.newvision.co.ug/news/637497-unclear-policies-hindering-e-procurement.html

Australian Government Department of Finance. (2014). Commonwealth procurement rules: Achieving value for money. Retrieved from http://www.finance.gov.au/ sites/default/files/2014%20Commonwealth%20Procurement%20Rules.pdf

Australian Government Department of Finance. (n.d.). Strategic guide to e-procurement. Retrieved from http://www.finance.gov.au/publications/strategic-guide-to-e-procurement/implementation.html

148

The Australian National Audit Office. (2010). Better practice guide on the strategic and operational management of assets by public sector entities. Retrieved from http://www.anao.gov.au/bpg_assets2010/HTML/index.html

Backus, M. (2001). E-government and developing countries: Introduction and examples. Retrieved from http://www.iicd.org/about/publications/egovernance-and-developing-countries-introduction-and-examples/report3.pdf

Bedi, K., Singh, P. J., & Srivastava, S. (2001). Government@net: New governance opportunities for India. New Delhi, India: Sage.

Bernard, H. R. (2000). Social research methods: Qualitative and quantitative approaches. Thousand Oaks, CA: Sage.

Callender, G., & Mathews, D. (2000). Government purchasing: An evolving profession? Journal of Public Budgeting, Accounting & Financial Management, 12(2), 272–290. Available from http://www.highbeam.com/doc/1P3-57959566.html

Callender, G., & Schapper, P. (2003). Public procurement reform in Western Australia: A federal-state evaluation. In L. Knight, C. Harland, J. Telgen, K. V. Thai, G. Callender, & K. McKen (Eds.), Public procurement: International cases and commentary (pp. 25–41). London, England: Routledge.

Carayannisa, E. G., & Popescu, D. (2005). Profiling a methodology for economic growth and convergence: Learning from the EU e-procurement experience for Central and Eastern European countries. Technovation, 25(1), 1–14. doi:10.1016/s0166-4972(03)00071-3

Carlson, M., & O’Neal-McElrath, T. (2008). Winning grants: Step by step. San Francisco, CA: Jossey-Bass.

Carter, L., & Bélanger, F. (2005). The utilization of e-government services: Citizen trust, innovation and acceptance factors. Information Systems Journal, 15(1), 15–25.doi:10.1111/j.1365-2575.2005.00183.x

Carter, R., & Grimm, R. (2001). Journal of Public Procurement under the FAU-NIGP partnership. Journal of Public Procurement, 1(1), 3–8. Retrieved from http://ippa.org/jopp/download/vol1/Carter%20&%20Grimm.pdf

Casaki, C., & Gelleri, P. (2005). Conditions and benefits of applying decision technological solutions as a tool to curb corruption within the procurement process: The case of Hungary. Journal of Purchasing and Supply Management, 11(5/6), 252–259. doi:10.1016/j.pursup.2006.01.002

149

Cheng, E. Y., & Wang, Y. J. (2006). Business process reengineering and ERP benefits. In Proceedings of the 11th Annual Conference of Asia Pacific Decision Sciences Institute (pp. 201–213). Retrieved from http://gebrc.nccu.edu.tw/proceedings/ APDSI/2006/201-213.pdf

Chvalkovská, J., & Skuhrovec, J. (2010). Measuring transparency in public spending: Case of Czech public e-procurement information system. Retrieved from http://ies.fsv.cuni.cz/default/file/download/id/13692

Commonwealth of Australia, Department of Finance and Deregulation, Business Procurement and Asset Management Group. (2013). Selling to the Australian government: A guide for business. Retrieved from http://www.finance.gov.au/ sites/default/files/A_guide_for_business.pdf?v=1

Concha, G., Astudillo, H., Porrua, M., & Pimenta, C. (2012, January). E-government procurement observatory, maturity model and early measurements. Government Information Quarterly, 29(Suppl. 1), S43–S50. doi:10.1016/j.giq.2011.08.005

Cook, P. F. (2010). How to perform a MANOVA in SPSS. Retrieved from http://www.ucdenver.edu/academics/colleges/nursing/Documents/PDF/MANOVAHowTo.pdf

Cooper, D., & Schindler, P. (2008). Business research methods. New York, NY: McGraw-Hill Irwin.

Curtin University of Technology. (2007). Multilateral Development Bank international survey of e-procurement system. Retrieved from http://www.unpcdc.org/media/ 2658/international%20survey%20of%20e-procurement%20systems.pdf

De Almeida, M. O. (2002). E-procurement by Brazil’s federal government. Retrieved from http://www.egov4dev.org/success/case/brazeproc.shtml

Deise, M. V., Nowikow, C., King, P., & Wright, A. (2000). Executive’s guide to e-business: From tactics to strategy. New York, NY: Wiley.

De Winter, J., Dodou, D., & Wieringa, P. (2009). Exploratory factor analysis with small sample sizes. Multivariate Behavioral Research, 44(2), 147–181. doi:10.1080/ 00273170902794206

Ermal, E., & Miranda, H. (2012, June). Efficiency evaluation of the public e-procurement system in the reduction of corruption: The Albanian case. Paper presented at the Transatlantic Conference on Transparency Research, Utrecht, Netherlands.

European Parliament and the Council. (2004). Directive 2004/18/EC of the European Parliament and of the Council. Official Journal of the European Union, 30(4), 114–240. Retrieved from http://eurocodes.jrc.ec.europa.eu/doc/ publicprocurementdirective.pdf

150

Evenett, S. J., & Hoekman, B. M. (2005). Government procurement: Market access, transparency, and multilateral trade rules. European Journal of Political Economy, 21(1), 163–183. doi:10.1016/j.ejpoleco.2004.01.001

Field, A. (2000). Discovering statistics using SPSS for Windows. London, England: Sage.

Fitzpatrick, J., Sanders, J., & Worthen, B. (2011). Program evaluation: Alternative approaches and practical guidelines. Upper Saddle River, NJ: Pearson Education.

Foulger, D. (1979, November). Credibility and the statistical interaction variable: Speaking up for multiplication as a source of understanding. Paper presented at the Annual Conference of the Speech Communication Association, San Antonio, TX. Retrieved from http://davis.foulger.info/papers/statisticalInteraction1979.htm

Fowler, F. (2002). Survey research methods (3rd ed.). Thousand Oaks, CA: Sage.

García-Santillán, A., Chávez, M. E. E., Boggero-Correa, P., & Vela-Aguilar, J. A. (2012). Student’s attitude toward computer and mathematics, interaction and engagement in the teaching-learning process: Empirical study on accounting, management, economy, international commerce and marketing undergraduate students. International Journal of Applied Science and Technology, 2(4), 1–11. Retrieved from http://www.ijastnet.com/journals/Vol_2_No_4_April_2012/1.pdf

Gardenal, F. (n.d.). Public e-procurement - Define, measure and optimize organizational benefits. Retrieved from http://www.ippa.org/IPPC4/Proceedings/05e-Procurement/Paper5-1.pdf

Garson, G. D. (2006). Syllabus for PA 765 - 766: Quantitative Research in Public Administration. Retrieved from http://faculty.chass.ncsu.edu/garson/PA765/ pa765syl.htm

General Services Administration. (n.d.). Federal Procurement Data System - Next Generation. Retrieved from https://www.fpds.gov/fpdsng_cms/index.php/en/

Grindle, M. (2010). Good governance: The inflation of an idea. Retrieved from http://ksghauser.harvard.edu/index.php/content/download/69011/1248854/version/1/file/Good_governance.pdf

Harris, R., & Rajora, R. (2006). Empowering the poor: Information and communication technologies for governance and poverty reduction: A study of rural development project in India. Retrieved from http://www.ethnosproject.org/wp-content/ uploads/EmpoweringThePoor.pdf

Heywood, J. B. (2002). E-procurement: Managing successful e-procurement implementation. Harlow, England: Prentice Hall Financial Times.

151

Hidayanto, A. N., Ditari, Y., & Chahyati, D. (2012, June). Study of e-procurement implementation impacts: A case study in PT. PLN. Paper presented at the International Conference on Management of Innovation and Technology 2012, Jakarta, Indonesia.

Hood, C., & Lodge, M. (2004). Competency, bureaucracy and public management reform: A comparative analysis. Governance, 17(3), 313–333. doi:10.1111/j. 0952-1895.2004.00248.x

Hui, W. S., Othman, R., Omar, N. H., Rahman, R. A., & Haron, N. H. (2011). Procurement issues in Malaysia. International Journal of Public Sector Management, 24(6), 567–593. doi:10.1108/09513551111163666

Iqbal, M. S., & Seo, J. W. (2008). E-government as an anti corruption tool: Korean cases. Journal of Korean Association for Regional Information Society, 11(2), 51–78. Retrieved from http://www.karis21.or.kr/karis21admin/adaction/down/03Iqbal_ Seo.pdf

International Monetary Fund. (1998). Ministers outline measures to strengthen monetary system; adopt code of good practices on fiscal transparency. [International Monetary Fund] IMF Survey, 27(8), 113–115. Retrieved from https://www.imf. org/external/pubs/ft/survey/pdf/042798.PDF

International Monetary Fund. (2010). World economic outlook: A survey by the staff of the International Monetary Fund. Washington, DC: Author.

Jeppesen, R. (2010). Accountability in public procurement: Transparency and the role of civil society. Retrieved from http://www.unpcdc.org/media/142496/story%20of %20an%20institution%20-%20accountability.pdf

Johnson, B., & Christensen, L. (2008). Educational research: Quantitative, qualitative and mixed approaches (3rd ed.). Thousand Oaks, CA: Sage.

Kaiser, F., & Rice, J. (1974). Little Jiffy, Mark IV. Educational and Psychological Measurement, 34(1), 111–117. doi:10.1177/001316447403400115

Kaufman, J. S., Crusto, C. A., Quan, M., Ross, E., Friedman, S. R., O’Rielly, K., & Call, S. (2006). Utilizing program evaluation as a strategy to promote community change: Evaluation of a comprehensive, community-based family violence initiative. American Journal of Community Psychology, 38(3/4), 191–200. doi:10.1007/s10464-006-9086-8

Kaufmann, D., Kraay, A., & Zoido-Lobaton, P. (1999). Governance matters. Washington, DC: World Bank.

152

Knight, L., Caldwell, N., Harland, C., & Telgen, J. (2003). Government reform and public procurement: Academic report of the first workshop. Retrieved from http://www.irspp.com/Local/pdfs/irspp1_academicreport.pdf

Kpundeh, S. (2004). Process interventions versus structural reforms: Institutionalizing anticorruption reforms in Africa. In B. Levy & S. Kpundeh (Eds.), Building state capacity in Africa: Learning from performance and results (pp. 257–282). Washington, DC: World Bank Institute Development Studies.

Ladipo, O., Sanchez, A., & Sopher, J. (2009). Accountability in public expenditures in Latin America and the Caribbean: Revitalizing reforms in financial management and procurement. Washington, DC: World Bank.

Laerd Statistics. (n.d.). Friedman test in SPSS. Retrieved from https://statistics.laerd.com/ spss-tutorials/friedman-test-using-spss-statistics.php

Lee, M. J., & Lee, S. J. (2012). The evaluation for value-oriented public e-procurement service. In Proceedings of International Public Procurement Conference (pp. 539–562). Retrieved from http://woppaa.org/wp-content/uploads/2012/09/ PAPER-II.15-THE-EVALUATION-FOR-VALUE-ORIENTED-PUBLIC-E-PROCUREMENT-SERVICE.pdf

Leipold, K. (2007, July). Electronic government procurement (e-GP) opportunities and challenges. Paper presented at the Modern Law for Global Commerce Congress to Celebrate the Fortieth Annual Session of the United Nations Commission on International Trade Law, Vienna, Austria. Retrieved from http://www.uncitral. org/pdf/english/congress/Leipold.pdf

Lynch, J. (n.d.). Procurement planning and the procurement plan: Why are they important? Retrieved from http://www.procurementclassroom.com/procurement-planning-and-the-procurement-plan-why-are-they-important/

Merriam, S. (1998). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass.

Mitra, R. K., & Gupta, M. P. (2007). Analysis of issues of e-governance in Indian police. Electronic Government: An International Journal, 4(1), 97–125. doi:10.1504/eg. 2007.012182

Morgan, L. L. (1996). Utilization-focused evaluation: A case study of a diversity program evaluation (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 9623545)

Morita, S., & Zaelke, D. (2005). Rule of law, good governance, and sustainable development. In Seventh International Conference on Environmental Compliance and Enforcement Proceedings (Vol. 1, pp. 15–21). Retrieved from http://www.inece.org/conference/7/vol1/05_Sachiko_Zaelke.pdf

153

Neupane, A., Soar, J., Vaidya, K., & Yong, J. (2012, August). Role of public e-procurement technology to reduce corruption in government procurement. Paper presented at the International Public Procurement Conference, Seattle, Washington. Retrieved from http://www.ippa.org/IPPC5/Proceedings/Part2/ PAPER2-8.pdf

Ng’ambi, D., & Brown, I. (2004). Utilisation-focused evaluation of ICT in education: The case of DFAQ consultation space. Educational Technology & Society, 7(3), 38–49. Retrieved from http://www.ifets.info/journals/7_3/5.pdf

Organization for Economic Co-operation and Development. (2005). Fighting corruption and promoting integrity in public procurement. Retrieved from http://www.keepeek.com/Digital-Asset-Management/oecd/governance/fighting-corruption-and-promoting-integrity-in-public-procurement_9789264014008-en#page1

Organization for Economic Co-operation and Development. (2006a). Methodology for assessment of national procurement systems. Retrieved from http://www.oecd. org/dataoecd/1/36/37130136.pdf

Organization for Economic Co-operation and Development. (2006b). Why is procurement important? Retrieved from http://unpcdc.org/media/6496/global%20procurement. pdf

Organization for Economic Co-operation and Development. (n.d.). E-procurement for good governance and development in Italy, North Africa and the Middle East. Retrieved from http://www.oecd.org/mena/governance/39856250.pdf

Osei-Tutu, E., Mensah, S., & Ameyaw, C. (2011, June). The level of compliance with the Public Procurement Act (ACT663) in Ghana. Paper presented at the Management and Innovation for a Sustainable Built Environment Conference, Amsterdam, The Netherlands.

Pashev, K., Dyulgerov, A., & Kaschiev, G. (2006). Corruption in public procurement: Risks and reform policies. Sofia, Bulgaria: Center for the Study of Democracy.

Patton, M. Q. (2012a). Essentials of utilization-focused evaluation. Thousand Oaks, CA: Sage.

Patton, M. Q. (2012b). A utilization-focused approach to contribution analysis. Evaluation, 18(3), 364–377. doi:10.1177/1356389012449523

Quesada, G., González, M. E., Mueller, J., & Mueller, R. (2010). Impact of e-procurement on procurement practices and performance. Benchmarking: An International Journal, 17(4), 516–538. doi:10.1108/14635771011060576

154

Rai, A., Tang, X., Brown, P., & Keil, M. (2006). Assimilation patterns in the use of electronic procurement innovations: A cluster analysis. Information and Management, 43(3), 336–349. doi:10.1016/j.im.2005.08.005

Ramirez, R., & Broadhead, D. (2013). Utilization focused evaluation: A primer for evaluators. Penang, Malaysia: Southbound.

Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York, NY: The Free Press.

Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York, NY: The Free Press.

Rotchanakitumnuai, S. (2010, December). Tacit knowledge sharing for good governance of e-government procurement. Paper presented at the 10th International Conference on Electronic Business, Shanghai, China.

Rotchanakitumnuai, S. (2013). The governance evidence of e-government procurement. Transforming Government: People, Process and Policy, 7(3), 309–321. doi:10.1108/TG-01-2013-0004

Russ-Eft, D. (1980). Validity and reliability in survey research. Palo Alto, CA: American Institutes for Research in the Behavioral Sciences.

Saidi, N., & Yared, H. (2002, April). E-government: Technology for good governance, development and democracy in the MENA countries. Paper submitted for the Mediterranean Development Forum IV: Fostering Digital Inclusion: The Role of ICT in Development, Amman, Jordan. Retrieved from http://www.passia.org/ goodgov/resources/eGovernment.pdf

Saxena, K. B. C. (2006). Toward excellence in e-governance. International Journal of Public Sector Management, 18(6), 498–513. doi:10.1108/09513550510616733

Schapper, P. R. (2008). The impact of e-procurement on corruption: The potential of e-procurement for curbing corruption risks. In Fighting bribery in public procurement in Asia and the Pacific: Proceedings of the Seventh Regional Seminar on Making International Anti-Corruption Standards Operational (pp. 113–127). Retrieved from http://browse.oecdbookshop.org/oecd/pdfs/product/ 2808041e.pdf

Schapper, P. R., Veiga Malta, J. N., & Gilbert, D. L. (2006). An analytical framework for the management and reform of public procurement. Journal of Public Procurement, 6(1/2/3), 1–26. Retrieved from http://ippa.org/jopp/download/vol6/ issue-1-2/Article_1_Malta.pdf

Schiavo-Campo, S., & Sundaram, P. (2000). To serve and preserve: Improving public administration in a competitive world. Manila, Philippines: Asian Development Bank.

155

Shakya, R. K. (2003). Theme paper on e-procurement. In Asia Foundation, Electronic procurement in His Majesty’s Government of Nepal (pp. 39-57). Retrieved from http://asiafoundation.org/publications/force-download.php?f=%2Fresources%2F pdfs%2Fnepalelectronic.pdf

Shakya, R. K. (2012). Procurement governance framework: Success to e-government procurement (E-GP) system implementation. Retrieved from http://woppaa.org/wp-content/uploads/2012/09/PAPER-II.16-PROCUREMENT-GOVERNANCE-FRAMEWORK.pdf

Shakya, R. K. (2014). E-government procurement (e-GP) strategy of Uganda. Kampala, Uganda: The Public Procurement and Disposal of Assets Authority.

Siddiquee, N. A., & Mohamed, M. Z. (2007, Fall). Paradox of public sector reforms in Malaysia: A good governance perspective. Public Administration Quarterly, 31(3/4), 284–312. Retrieved from ABI/INFORM Global database.

Smith, P. (n.d.). Implementing the eprocurement mandate: Technology choices and key decision factors. Retrieved from http://portugal.vortal.biz/files/Brochuras_Online/ WhitePaper_SpendMatters_Dec14.pdf

Steyaert, S., & Lisoir, H. (2005). Participatory methods toolkit: A practitioner’s manual. Brussels, Belgium: King Baudouin Foundation and the Flemish Institute for Science and Technology Assessment.

Stringer, E. T. (2007). Action research (3rd ed.). Thousand Oaks, CA: Sage.

Stufflebeam, D. (1999). Program evaluations metaevaluation checklist. Retrieved from http://www.wmich.edu/evalctr/archive_checklists/program_metaeval.pdf

Suchman, A. E. (1977). Evaluating educational programs. In F. G. Caro (Ed.), Readings in evaluation research (pp. 43–48). New York, NY: Russell Sage Foundation.

Tassabehji, R., & Moorhouse, A. (2008). The changing role of procurement: Developing professional effectiveness. Journal of Purchasing and Supply Management, 14(1), 516–538. doi:10.1016/j.pursup.2008.01.005

Thai, K. V. (2001). Public procurement re-examined. Journal of Public Procurement, 1(1), 9–50. Retrieved from http://ippa.org/jopp/download/vol1/Thai.pdf

Thai, K. (2004). Introduction to public procurement. Seattle, WA: National Institute of Government Purchasing.

Trepte, P. (1998). Procurement regulation and emerging economies: The examples of Laos and Bhutan. In S. Arrowsmith & A. Davies (Eds.), Public procurement: Global revolution (pp. 97–118). London, England: Kluwer Law International.

156

Trkman, P., & McCormack, K. (2010). Estimating the benefits and risks of implementing e-procurement. IEEE Transactions on Engineering Management, 57(2), 338–349. doi:10.1109/tem.2009.2033046

United Nations. (2011). E-procurement: Towards transparency and efficiency in public service delivery. Retrieved from http://unpan1.un.org/intradoc/groups/public/ documents/un/unpan047627.pdf

United Nations Economic and Social Commission for Asia and the Pacific. (2007). Economic and social development in Asia and the Pacific. Retrieved from http://www.unescap.org/sites/default/files/good-governance.pdf

United Nations Economic and Social Commission for Asia and the Pacific. (2009). What is good governance? Retrieved from http://www.unescap.org/sites/default/files/ good-governance.pdf

United Nations Educational, Scientific and Cultural Organization. (2005). E-government toolkit for developing countries. Retrieved from http://omec.uab.cat/Documentos/ mitjans_dem_gov/0138.pdf

Vaidya, K., Callender, G., Sajeev, A. S. M., & Gao, J. (2004, April). Toward a model for measuring the performance of e-procurement initiatives in the Australian public sector: A Balanced Scorecard approach. Paper presented at the Australian Electronic Governance Conference, Melbourne. Retrieved from http://www.unpcdc.org/media/5184/australia%20measuring%20performance.pdf

Vajpey, A. (2010). Fair and transparent construction procurement with new ISO standards. Retrieved from http://developmentprocurement.com/fair-transparent-construction-procurement-with-new-iso-standards/

Veiga Malta, J. N., Schapper, P., Calvo-Gonzalez, O., & Berroa, D. (2011). Old rules, new realities: Are existing public procurement systems addressing current and future needs? Retrieved from http://siteresources.worldbank.org/ EXTLACOFFICEOFCE/Resources/Old_Rules_New_Reality_Sept2011_Web.pdf

Walker, J., & Madden, S. (2009). Statistics in criminology and criminal justice: Analysis and interpretation (3rd ed.). Burlington, MA: Jones & Bartlett.

Weiss, C. H. (2000). Which links in which theories shall we evaluate? Program theory in evaluation: Challenges and opportunities. New Directions for Evaluation, 87, 35–45. doi:10.1002/ev.1180

Wittig, W. A. (n.d.). Public procurement and the development agenda. Retrieved from http://www.wto.org/english/tratop_e/gproc_e/wkshop_tanz_jan03/itcdemo1_e.pdf

Wittig, W. A. (1998). Report of the Conference on Public Procurement Reform in Africa. Geneva, Switzerland: International Trade Centre.

157

World Bank. (1992). Governance and development. Retrieved from http://www-wds. worldbank.org/external/default/WDSContentServer/WDSP/IB/1999/09/17/000178830_98101911081228/Rendered/PDF/multi_page.pdf

World Bank. (2013). Project signing: Bangladesh receives additional $34.5 million for digitizing public procurement in Bangladesh [Press release]. Retrieved from http://www.worldbank.org/en/news/press-release/2013/06/10/project-signing-bangladesh-receives-additional-34-5-million-for-digitizing-public-procurement-in-bangladesh

Zaiontz, C. (n.d.). Real statistics using Excel: Cronbach’s alpha. Retrieved from http://www.real-statistics.com/reliability/cronbachs-alpha/

158

APPENDIX A. RECOMMENDED E-GOVERNMENT PROCUREMENT SYSTEM FEATURES

Transparency

Availability of the following facilities (list is based on the international best

practices) on the e-procurement portal are all vital ingredients of transparency in the e-

procurement system contributing to enhancing current and future good governance efforts

in public procurement sector:

• Public Procurement Act and Public Procurement Regulations as well as other circulars and directives

• Mandatory publication of annual procurement plan by the public procuring entities

• Publication of invitation to bid with eligibility, qualification, evaluation, and preferential criteria for bidding process, or access to the bidding documents

• Ensuring availability of budget for the procurement for public entity

• Provide traceability of transactions through audit trails for the benefit of public entities and regulatory/oversight agencies

• Publication of amendment to bidding document

• Sharing clarification to queries on bidding document

• Sharing bid opening records

• Status of the different stages of the procurement process

• Publication of report on the best evaluated bidders

• Publication of contract award details

• Opportunity to request debriefing of the reasons of losing the contract

159

• Lodging complaint on the process and practice of procurement and sharing the review decision on such complaints

• Publication of contract implementation progress along with variations and completion

• Online public forum for public discussions on public procurement

• Contractor, supplier, and consultant appraisal information

• Publication of procurement performance statistics and analytics

Accountability

Availability of the following facilities (list is based on the international best

practices) on the e-procurement portal are all vital ingredients of accountability in the e-

procurement system contributing to enhancing current and future good governance efforts

in public procurement sector:

• Keeping records of all reviews, decisions, approval and rejection comments at the stages of public procurement process under the agreed workflow and based on the roles and authorities of the actors in the process

• Keeping records of prequalification and postqualification of bidders, negotiation with bidders

• Publication of contract award notice on time

• Ensuring quality of the deliveries before making payments to the contractors

• Ensuring the budget for the procurement and payments before publishing invitation to bids, or before awarding a contract

• Details of payment transactions in adherence to the terms of payment agreed upon in contract

• Contract amendments details with reasons

• Timely completion of actions and decisions required by the actors in procurement process

160

• Responsiveness of the authorized actors of the process on timely handling complaints submitted by the bidders, and disputes during contract execution

Corruption Control

Availability of the following facilities (list is based on the international best

practices) on the e-procurement portal are all vital ingredients of corruption control in the

e-procurement system contributing to enhancing current and future good governance

efforts in public procurement sector:

• Minimizing face-to-face interaction between public procurement entity users and bidders

• Keeping bidder information anonymous to all users until the bid opening time

• Listing of debarred suspended bidders from procurement processes

• Disclosure of relationship with the actors involved in the procurement process

• Automated control of using discretion and unauthorized judgment by the authorized users in making decisions

• Mandatory binding of the users by a disclosure of their noninvolvement in any kind of fraudulent and corruption activities

• Implementation of e-contract management function

• Application of four-eyes principle in all critical decision processes

Efficiency, Effectiveness, and Predictability

Availability of the following facilities (list is based on the international best

practices) on the e-procurement portal are all vital ingredients of efficiency,

effectiveness, and predictability in the e-procurement system contributing to enhancing

current and future good governance efforts in public procurement sector:

161

• Most of the compliance to Public Procurement Act, regulations, and guidelines are automatically validated by the e-procurement system in key stages of compliance requirements

• Procurements are carried out in adherence to the annual procurement plan and procurements are initiated on time

• Standard bidding documents and contract templates are available

• Clearly defined roles and authority of the public procurement officers, and simplified automated administrative workflow available

• Business process reengineering carried out and procurement processes and practices are simplified

• Time for getting clarifications on bidding document from public procuring entity is reduced

• Evaluation processes are assisted by semiautomated or fully automated tools in e-procurement system

• Time for evaluation is reduced

• If processing time is delayed by any official, system sends timely reminders

• Statistics and analytics on public procurement performance of public procurement entities are generated against the performance indicators, and provides pointers for improvements

• Review and audit of the procurement incidences are easier

• Contract processing and decision on awarding time is reduced

• Contract management has been easier

• Payments to contractors are tracked and paid on timely manner

• Contract variation practices have been reduced, and projects are completed on time, within the budget and with expected quality

• Complaints and disputes have been reduced

162

• Bidders are provided with procurement information and procurement plans upfront, and adequately get time to plan, and prepare required resources

• Enhanced predictability on decision timings

• Ensured value-for-money

Easy Access to Procurement Information and Instruments

Availability of the following facilities (list is based on the international best

practices) on the e-procurement portal are all vital ingredients of easy access to the

Procurement processes, information and instruments in the e-procurement system

contributing to enhancing current and future good governance efforts in public

procurement sector:

• Provides easy access to Public Procurement Act, regulations, circulars, directives, manuals, and e-government procurement guidelines

• Browsing speed of the e-procurement system is satisfactory

• E-procurement system does not require high Internet bandwidth

• High availability of the e-procurement system supported by failover data center arrangements

• Registration of users to use the e-procurement system is instant

• Registration of users in the e-procurement system is free of cost or considerably affordable in comparison to value receivable through the e-procurement system

• Provides preview of the bidding document free of cost along with eligibility, qualification, preferential, and evaluation criteria before buying the document,

• Provides features of reusing bidding documents for procuring entities and bid documents for bidders with minimum changes

• Provides easy access to features to lodge complaints on the procurement process

163

• Provides online forms as well as offline provisions for preparing bidding documents as well as bid offer preparation

• Provides easy access to procurement-related documents and decisions for public procuring entities

• Above 70% of the procurement processes of the procurement cycle are automated

Rule of Law and Equal Treatment to the Bidders

Availability of the following facilities (list is based on the international best

practices) on the e-procurement portal are all vital ingredients of the enforcement of rule

of law and creating environment for the equal treatment to the bidders in the e-

procurement system contributing to enhancing current and future good governance efforts

in public procurement sector:

• Provides templates of standard bidding documents are reengineered and made available fully in line with the prevalent Public Procurement Act, Public Procurement Regulations, guidelines, and other directives

• Provides automated compliance verification at each stage of procurement of required procurement methods, processes, thresholds, rules, number of days for specific events, and so on, based on the nature of procurement as provisioned in the act and regulation

• Provides administrative workflow based on role and authority of the public procurement entity officers and other users

• Allows submission of bid only if all requested documents are uploaded and information is provided to make the submitted bid responsive

• Provides bidders equal treatment by keeping bidders’ names and addresses anonymous in the system until the time of bid opening

• Provides access to e-procurement system through different channels of access such as web browsers and mobile phones

• Enhances level of citizen trust in public procurement

164

Civil Society Awareness

Availability of the following facilities (list is based on the international best

practices) on the e-procurement portal are all vital ingredients for the making the civil

society (i.e., general public) aware about the information on the public procurements,

government expenditure on development projects, details on the use of taxpayers’ money,

and also about the contribution of civil society as a watchdog in public procurement:

• Publication of annual procurement plan with details on the procurement being planned, expected timeline for the start of procurement and completion of the contract, use of competitive bidding procedures, estimated cost of the procurement, and so on, in line with the open data guidelines

• Publication of invitation to bids with contractor selection criteria

• Contract award information in line with the Open Contract Guidelines with the details on the winning contractor, contract amount, agreed terms with the contractors, contract execution site information

• Publishing the variation of contracts and their reasons

• Disclosure of complaints, and their resolutions

• Public discussion forum for the discussion on the issues related to procurements

• Publication of statistics and analytics on the public procurement performance of the procuring entities

• Establishment of helpdesk for answering queries from the general public

• Provision to provide forensic information in relation to procurements for the auditing and other oversight agencies for investigation in case of any suspected fraud and corruption cases.

165

APPENDIX B. VALIDITY AND RELIABILITY ANALYSIS TABLES Validity and reliability data for good governance factors are summarized in Table

2. The following tables provide validity and reliability data per good governance factor.

The extraction method for all communalities and total variance explained values was

principal component analysis.

Transparency

Table B1. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Transparency

Kaiser–Meyer–Olkin measure of sampling adequacy .471

Bartlett’s test of sphericity Approx. chi square 595.004 df 276 Sig. .000

166

Table B2. Communalities for Transparency

Survey Question No. Initial Extraction

1 1.000 .583

2 1.000 .683

3 1.000 .763

4 1.000 .779

5 1.000 .819

6 1.000 .780

7 1.000 .794

8 1.000 .636

9 1.000 .668

10 1.000 .814

11 1.000 .764

12 1.000 .717

13 1.000 .769

14 1.000 .783

15 1.000 .658

16 1.000 .639

17 1.000 .823

18 1.000 .580

19 1.000 .714

20 1.000 .843

21 1.000 .728

22 1.000 .842

23 1.000 .728

24 1.000 .708

167

Table B3. Total Variance Explained for Transparency

Survey Question

No.

Initial eigenvalue

Extraction sum of squared loadings

Rotation sum of squared loadings

Total % of var. Cum % Total % of var. Cum % Total

1 4.701 19.587% 19.587% 4.701 19.587% 19.587% 3.536

2 3.089 12.872% 32.460% 3.089 12.872% 32.460% 2.574

3 2.880 12.000% 44.459% 2.880 12.000% 44.459% 2.307

4 1.667 6.945% 51.404% 1.667 6.945% 51.404% 2.142

5 1.550 6.460% 57.864% 1.550 6.460% 57.864% 1.929

6 1.414 5.891% 63.755% 1.414 5.891% 63.755% 1.846

7 1.193 4.972% 68.726% 1.193 4.972% 68.726% 1.677

8 1.123 4.679% 73.405% 1.123 4.679% 73.405% 1.606

9 .991 4.130% 77.536%

10 .789 3.285% 80.821%

11 .715 2.978% 83.799%

12 .625 2.603% 86.402%

13 .579 2.413% 88.815%

14 .483 2.014% 90.829%

15 .450 1.874% 92.703%

16 .357 1.489% 94.192%

17 .335 1.396% 95.588%

18 .272 1.133% 96.721%

19 .220 .916% 97.637%

20 .184 .765% 98.402%

21 .147 .614% 99.016%

22 .105 .438% 99.454%

23 .081 .336% 99.790%

24 .050 .210% 100.000%

168

Accountability

Table B4. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Accountability

Kaiser–Meyer–Olkin measure of sampling adequacy .819

Bartlett’s test of sphericity Approx. chi square 550.499 df 105 Sig. .000

Table B5. Communalities for Accountability

Survey Question No. Initial Extraction

1 1.000 .701

2 1.000 .776

3 1.000 .630

4 1.000 .601

5 1.000 .722

6 1.000 .692

7 1.000 .719

8 1.000 .772

9 1.000 .736

10 1.000 .661

11 1.000 .736

12 1.000 .632

13 1.000 .779

14 1.000 .618

15 1.000 .733

169

Table B6. Total Variance Explained for Accountability

Survey Question

No.

Initial eigenvalue Extraction sum of squared loadings

Rotation sum of squared loadings

Total % of

variance Cumulative

% Total % of

variance Cumulative

% Total

1 6.988 46.588% 46.588% 6.988 46.588% 46.588% 5.516

2 2.305 15.370% 61.957% 2.305 15.370% 61.957% 2.748

3 1.213 8.088% 70.046% 1.213 8.088% 70.046% 2.242

4 .904 6.030% 76.076%

5 .809 5.393% 81.468%

6 .634 4.227% 85.695%

7 .492 3.282% 88.978%

8 .401 2.676% 91.653%

9 .293 1.952% 93.606%

10 .254 1.694% 95.300%

11 .234 1.560% 96.860%

12 .152 1.013% 97.873%

13 .146 .974% 98.847%

14 .098 .652% 99.499%

15 .075 .501% 100.000%

Corruption Control

Table B7. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Corruption Control

Kaiser–Meyer–Olkin measure of sampling adequacy .748

Bartlett’s test of sphericity Approx. chi square 90.081 df 6 Sig. .000

170

Table B8. Communalities for Corruption Control

Survey Question No. Initial Extraction

1 1.000 .429

2 1.000 .828

3 1.000 .819

4 1.000 .555

Table B9. Total Variance Explained for Corruption Control

Survey Question

No.

Initial eigenvalue Extraction sum of squared loadings

Total % of variance Cumulative % Total % of variance Cumulative %

1 2.631 65.783% 65.783% 2.631% 65.783% 65.783%

2 .736 18.407% 84.190%

3 .452 11.296% 95.485%

4 .181 4.515% 100.000%

Efficiency, Effectiveness, and Predictability

Table B10. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Efficiency, Effectiveness, and Predictability

Kaiser–Meyer–Olkin measure of sampling adequacy .828

Bartlett’s test of sphericity Approx. chi square 128.263 df 28 Sig. .000

171

Table B11. Communalities for Efficiency, Effectiveness, and Predictability

Survey Question No. Initial Extraction

1 1.000 .484

2 1.000 .542

3 1.000 .671

4 1.000 .610

5 1.000 .763

6 1.000 .667

7 1.000 .571

8 1.000 .404

Table B12. Total Variance Explained for Efficiency, Effectiveness, and Predictability

Survey Question

No.

Initial eigenvalue

Extraction sum of squared loadings

Rotation sum of squared loadings

Total % of

variance Cumulative

% Total % of

variance Cumulative

% Total

1 3.604 45.053% 45.053% 3.604 45.053% 45.053% 3.603

2 1.108 13.845% 58.898% 1.108 13.845% 58.898% 1.109

3 .926 11.580% 70.478%

4 .730 9.123% 79.601%

5 .544 6.801% 86.402%

6 .447 5.594% 91.995%

7 .337 4.209% 96.204%

8 .304 3.796% 100.000%

172

Easy Access

Table B13. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Easy Access

Kaiser–Meyer–Olkin measure of sampling adequacy .633

Bartlett’s test of sphericity Approx. chi square 73.029 df 28 Sig. .000

Table B14. Communalities for Easy Access

Survey Question No. Initial Extraction

1 1.000 .692

2 1.000 .630

3 1.000 .672

4 1.000 .648

5 1.000 .597

6 1.000 .677

7 1.000 .588

8 1.000 .603

173

Table B15. Total Variance Explained for Easy Access

Survey Question

No.

Initial eigenvalue

Extraction sum of squared loadings

Rotation sum of squared loadings

Total % of

variance Cumulative

% Total % of

variance Cumulative

% Total

1 2.417 30.209% 30.209% 2.417 30.209% 30.209% 2.226

2 1.568 19.595% 49.804% 1.568 19.595% 49.804% 1.605

3 1.124 14.049% 63.853% 1.124 14.049% 63.853% 1.277

4 .909 11.362% 75.214%

5 .642 8.020% 83.234%

6 .540 6.756% 89.990%

7 .413 5.166% 95.156%

8 .388 4.844% 100.000%

Rule of Law and Equity

Table B16. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Rule of Law and Equity

Kaiser–Meyer–Olkin measure of sampling adequacy .637

Bartlett’s test of sphericity Approx. chi square 69.793 df 21 Sig. .000

174

Table B17. Communalities for Rule of Law and Equity

Survey Question No. Initial Extraction

1 1.000 .486

2 1.000 .509

3 1.000 .606

4 1.000 .409

5 1.000 .816

6 1.000 .624

7 1.000 .364

Table B18. Total Variance Explained for Rule of Law and Equity

Survey Question

No.

Initial eigenvalue

Extraction sum of squared loadings

Rotation sum of squared loadings

Total % of

variance Cumulative

% Total % of

variance Cumulative

% Total

1 2.630 37.572% 37.572% 2.630 37.572% 37.572% 2.203

2 1.183 16.901% 54.473% 1.183 16.901% 54.473% 1.610

3 .945 13.504% 67.977%

4 .777 11.106% 79.083%

5 .693 9.893% 88.976%

6 .431 6.162% 95.138%

7 .340 4.862% 100.000%

175

Civil Society Awareness

Table B19. Kaiser–Meyer–Olkin Measure and Bartlett’s Test Data for Civil Society Awareness

Kaiser–Meyer–Olkin measure of sampling adequacy. .752

Bartlett’s test of sphericity Approx. chi square 129.078 df 6 Sig. .000

Table B20. Communalities for Civil Society Awareness

Survey Question No. Initial Extraction

1 1.000 .007

2 1.000 .907

3 1.000 .885

4 1.000 .868

Table B21. Total Variance Explained for Civil Society Awareness

Survey Question

No.

Initial eigenvalues Extraction sums of squared loadings

Total % of variance Cumulative % Total % of variance Cumulative %

1 2.667 66.682% 66.682% 2.667 66.682% 66.682%

2 .998 24.947% 91.628%

3 .204 5.090% 96.719%

4 .131 3.281% 100.000%

176

APPENDIX C. FRIEDMAN/CHI-SQUARE TEST ANALYSIS AND MULTIVARIATE ANALYSIS TABLES

Transparency

Table C1. Friedman Test Results for Transparency

N 52

Chi square 430.655

df 23

Asymp. sig. .000

177

Table C2. Friedman Chi-Square for Goodness-for-Fit Test Results for Transparency

Survey Question No. Chi square df Asymp. sig.

1 64.731a 4 .000

2 38.231b 5 .000

3 22.077b 5 .000

4 44.923c 3 .000

5 38.923c 3 .000

6 37.538c 3 .000

7 40.769c 3 .000

8 64.154a 4 .000

9 40.308a 4 .000

10 52.769a 3 .000

11 17.615b 4 .001

12 36.846b 5 .000

13 15.115c 4 .004

14 20.115c 4 .000

15 12.615c 3 .006

16 8.769c 3 .033

17 26.077a 4 .000

18 64.538a 4 .000

19 32.038a 4 .000

20 35.231b 3 .000

21 28.308b 5 .000

22 30.500c 4 .000

23 10.500c 4 .033

24 25.692c 4 .000

aZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 10.4. bZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 8.7. cZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 13.0.

178

Accountability

Table C3. Friedman Test Results for Accountability

N 52

Chi square 195.092

df 14

Asymp. sig. .000

Table C4. Friedman Chi-Square for Goodness-for-Fit Test Results for Accountability

Survey Question No. Chi square df Asymp. sig.

1 38.192a 4 .000

2 25.885a 4 .000

3 25.115a 4 .000

4 37.231a 4 .000

5 22.038a 4 .000

6 32.000b 5 .000

7 47.231b 5 .000

8 21.077a 4 .000

9 20.115a 4 .000

10 14.346a 4 .006

11 8.385a 4 .078

12 12.423a 4 .014

13 22.808a 4 .000

14 16.308a 3 .001

15 24.346b 4 .000

aZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 10.4. bZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 8.7.

179

Corruption Control

Table C5. Friedman Test Results for Corruption Control

N 52

Chi square 3.069

df 3

Asymp. sig. .381

Table C6. Friedman Chi-Square for Goodness-for-Fit Test Results for Corruption Control

Survey Question No.

1 2 3 4

Chi square 17.423a 11.269a 24.385b 20.923b

df 4 4 5 5

Asymp. sig. .002 .024 .000 .001

aZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 10.4. bZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 8.7.

Efficiency, Effectiveness, and Predictability

Table C7. Friedman Test Results for Efficiency, Effectiveness, and Predictability

N 52

Chi square 61.423

df 7

Asymp. sig. .000

180

Table C8. Friedman Chi-Square for Goodness-for-Fit Test Results for Efficiency, Effectiveness, and Predictability

Survey Question No.

1 2 3 4 5 6 7 8

Chi square 40.462a 23.769b 41.692a 39.731b 42.808b 51.654b 35.500b 56.462c

df 3 4 3 4 4 4 4 5

Asymp. sig. .000 .000 .000 .000 .000 .000 .000 .000

aZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 10.4. bZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 8.7. cZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 13.0.

Easy Access

Table C9. Friedman Test Results for Easy Access

N 52

Chi square 99.792

df 7

Asymp. sig. .000

Table C10. Friedman Chi-Square for Goodness-for-Fit Test Results for Easy Access

Survey Question No.

1 2 3 4 5 6 7 8

Chi square 37.308a 72.808b 21.154a 41.077c 13.192b 32.000a 90.308b 22.808b

df 5 4 5 3 4 5 4 4

Asymp. sig. .000 .000 .001 .000 .010 .000 .000 .000

aZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 10.4. bZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 8.7. cZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 13.0.

181

Rule of Law and Equity

Table C11. Friedman Test Results for Rule of Law and Equity

N 52

Chi square 35.992

df 6

Asymp. sig. .000

Table C12. Friedman Chi-Square for Goodness-for-Fit Test Results for Rule of Law and Equity

Survey Question No.

1 2 3 4 5 6 7

Chi square 128.462a 82.615b 80.462b 45.692b 136.269c 26.000d 50.308c

df 5 3 3 3 4 2 4

Asymp. sig. .000 .000 .000 .000 .000 .000 .000 aZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 10.4. bZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 8.7. cZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 13.0. dZero cells (0.0%) have expected frequencies less than 5. The minimum expected cell frequency is 17.3.

Civil Society Awareness

Table C13. Friedman Test Results for Civil Society Awareness

N 52

Chi square 41.699

df 3

Asymp. sig. .000

182

Table C14. Friedman Chi-Square for Goodness-for-Fit Test Results for Civil Society Awareness

Survey Question No.

1 2 3 4

Chi squarea 43.192 19.923 13.385 20.692

df 4 4 4 4

Asymp. sig. .000 .001 .010 .000

aZero cells (0.0%) have expected frequencies less than 5; the minimum expected cell frequency is 10.4.

183

Multivariate Analysis of Variance

Table C15. Multivariate Analysis of Variance: Descriptive Statistics

Construct Respondent Mean Std. deviation N

Transparency Bidder 3.1673 .44177 11 Public entity 3.3134 .44267 35 Regulatory/oversight 3.4817 .51882 6 Total 3.3019 .45066 52

Accountability Bidder 2.8236 .84872 11 Public entity 2.7357 .79579 35 Regulatory/oversight 2.7883 .68464 6 Total 2.7604 .78151 52

Corruption Control Bidder 2.7955 1.02967 11 Public entity 2.6357 .96509 35 Regulatory/oversight 2.4167 .93095 6 Total 2.6442 .96170 52

Efficiency, Effectiveness, and Predictability

Bidder 3.7864 .61907 11 Public entity 3.2989 .66130 35 Regulatory/oversight 3.8550 .42208 6 Total 3.4662 .66578 52

Easy Access Bidder 3.5827 .58378 11 Public entity 3.0023 .59570 35 Regulatory/oversight 3.2100 .52805 6 Total 3.1490 .62171 52

Rule of Law and Equity Bidder 3.8573 .39852 11 Public entity 4.0209 .47146 35 Regulatory/oversight 4.1417 .31403 6 Total 4.0002 .44277 52

Civil Society Awareness Bidder 3.1364 .83937 11 Public entity 2.5714 .82598 35 Regulatory/oversight 2.3750 .75416 6 Total 2.6683 .84413 52

184

Table C16. Multivariate Analysis of Variance: Tests

Effect Value F Hypothesis

df Error

df Sig.

Partial eta

squared Noncent. parameter

Observed power

Intercept Pillai’s trace

.987 483.144a 7.000 43.000 .000 .987 3382.009 1.000a

Wilk’s lambda

.013 483.144a 7.000 43.000 .000 .987 3382.009 1.000a

Hotelling’s trace

78.651 483.144a 7.000 43.000 .000 .987 3382.009 1.000a

Roy’s largest root

78.651 483.144a 7.000 43.000 .000 .987 3382.009 1.000a

Respondent Pillai’s trace

.527 2.250 14.000 88.000 .012 .264 31.502 .954

Wilk’s lambda

.525 2.336a 14.000 86.000 .009 .276 32.707 .961a

Hotelling’s trace

.806 2.418 14.000 84.000 .007 .287 33.854 .967

Roy’s largest root

.654 4.113 7.000 44.000 .001 .396 28.794 .972

aThe statistic is an upper bound on F that yields a lower bound on the significance level.

185

Table C17. Multivariate Analysis of Variance: Tests of Between-Subjects Effects

Source Dependent

variable

Type III sum of squares df

Mean square F Sig.

Partial eta

squared Noncent. parameter

Observed power

Corrected model

Transparency .398a 2 .199 .979 .383 .038 1.958 .211a

Accountability .070b 2 .035 .055 .946 .002 .110 .058b

Corruption Control

.565c 2 .282 .297 .744 .012 .594 .095c

Efficiency, Effectiveness, and Predictability

3.015d 2 1.507 3.770 .030 .133 7.540 .661d

Easy Access 2.845e 2 1.423 4.132 .022 .144 8.265 .704e

Rule of Law and Equity

.360f 2 . .914 .407 .036 1.829 .199f

Civil Society Awareness

3.255g 2 1.627 2.410 .100 .090 4.820 .463g

Intercept Transparency 346.845 1 346.845 1706.386 .000 .972 1706.386 1.000

Accountability 243.524 1 243.524 383.952 .000 .887 383.952 1.000

Corruption Control

215.234 1 215.234 226.302 .000 .822 226.302 1.000

Efficiency, Effectiveness, and Predictability

418.276 1 418.276 1046.119 .000 .955 1046.119 1.000

Easy Access 335.290 1 335.290 974.019 .000 .952 974.019 1.000

Rule of Law and Equity

504.899 1 504.899 2566.776 .000 .981 2566.776 1.000

Civil Society Awareness

228.314 1 228.314 338.135 .000 .873 338.135 1.000

Respondent Transparency .398 2 .199 .979 .383 .038 1.958 .211

Accountability .070 2 .035 .055 .946 .002 .110 .058

Corruption Control

.565 2 .282 .297 .744 .012 .594 .095

186

Table C17. Multivariate Analysis of Variance: Tests of Between-Subjects Effects (continued)

Source Dependent

variable

Type III sum of squares df

Mean square F Sig.

Partial eta

squared Noncent. parameter

Observed power

Respondent (continued)

Efficiency, Effectiveness, and Predictability

3.015 2 1.507 3.770 .030 .133 7.540 .661

Easy Access 2.845 2 1.423 4.132 .022 .144 8.265 .704

Rule of Law and Equity

.360 2 .180 .914 .407 .036 1.829 .199

Civil Society Awareness

3.255 2 1.627 2.410 .100 .090 4.820 .463

Error Transparency 9.960 49 .203

Accountability 31.079 49 .634

Corruption Control

46.603 49 .951

Efficiency, Effectiveness, and Predictability

19.592 49 .400

Easy Access 16.867 49 .344

Rule of Law and Equity

9.639 49 .197

Civil Society Awareness

33.086 49 .675

Total Transparency 577.298 52

Accountability 427.374 52

Corruption Control

410.750 52

Efficiency, Effectiveness, and Predictability

647.346 52

Easy Access 535.368 52

187

Table C17. Multivariate Analysis of Variance: Tests of Between-Subjects Effects (continued)

Source Dependent

variable

Type III sum of squares df

Mean square F Sig.

Partial eta

squared Noncent. parameter

Observed power

Total (continued)

Rule of Law and Equity

842.078 52

Civil Society Awareness

406.563 52

Corrected total

Transparency 10.358 51

Accountability 31.149 51

Corruption Control

47.168 51

Efficiency, Effectiveness, and Predictability

22.607 51

Easy Access 19.712 51

Rule of Law and Equity

9.998 51

Civil Society Awareness

36.340 51

aR squared = .038 (adjusted R squared = −.001). bR squared = .002 (adjusted R squared = −.038). cR squared = .012 (adjusted R squared = −.028). dR squared = .133 (adjusted R squared = .098). eR squared = .144 (adjusted R squared = .109). fR squared = .036 (adjusted R squared = −.003). gR squared = .090 (adjusted R squared = .052).

188

APPENDIX D. STUDY FEEDBACK AND RECOMMENDATIONS

Step no. Good governance aspect observation Recommendation

1 Transparency 1.1 Mandatory to publish annual

procurement plan. Annual procurement plan should be possible to update. Not clear in annual procurement plan whether budget is secured for the procurement.

Should be updated in quarterly basis if any changes occurred in approved annual procurement plan. Should include procurement information on both the electronic procurement as well as traditional information. Budget should be ensured and status of budget approval should be published in annual procurement plan.

1.2 Traceability of transaction is not adequate.

Full audit trail of the all transactions of all users should be recorded and traceable

1.3 Bid opening record is not shared with the bidders and also not published in the portal.

Bid opening record should be shared automatically with the bidders through e-mail, when such record is prepared at the time of bid opening.

1.4 Debriefing of contract award is not available.

There should be an online debriefing facility to the bidders who lost the contract if requested.

1.5 Best evaluated bidders are not published.

E-procurement portal should publish best evaluated bidders when it issues notification of contract awards.

1.6 There are no guidelines published for using and authenticating the transactions made in the e-procurement portal.

An e-procurement guidelines, equivalent to Public Procurement Regulations, should be developed and published in the e-procurement portal providing clear strategy for implementation, terms and conditions of use, disclaimer, and authenticity of the actions taken and data records in the e-procurement portal as well as provide validity to the changes made in processes and practices to the traditional procurement processes and practices.

1.7 Public discussion forum is not available.

There should be an open platform for the discussions on procurement issues online.

1.8 Not linked with any social media. As the trend and impact of social media is tremendously high on citizen and stakeholder engagement, e-procurement portal needs social media enablement linking with Facebook, LinkedIn, and Twitter to disseminate the procurement opportunities, as well as public-procurement-related information.

1.9 Public procurement performance analytics not published.

E-procurement portal should publish statistics and analytics on the public procurement performance of the public procuring entities in detail in regular basis for public awareness.

1.10 Complaint review information not published.

E-procurement portal should publish the lodged complaints and the outcome of the review of the complaints for public awareness.

1.11 Not enabled mobile-based Short Message Service alerts.

E-procurement system should also use the mobile Short Message Service for sending alerts of opportunities and important events in the portal on a subscription basis.

189

Step no. Good governance aspect observation Recommendation

1.12 Trend of low bidding is challenging for quality execution of the contracts. Low bidding raised some challenges: (a) contractors compromise on quality, (b) contractors quit contract during its execution, (c) contract remains incomplete, and (d) increase in contract variations.

To resolve this challenges, some options are recommended: (a) Disclose the estimated cost of the procurement; (b) To control at the time of bid submission, if the bid is substantially low. Permissible minimum offer can be set in the system, but for this public procurement regulatory and monitoring agency should work out on the reliable price index to be used for preparing the estimate of the procurements; and (c) Increase the weight substantially in quality criteria, giving less weight to price.

2 Accountability 2.1 There is no code of conduct for all the

actors of the procurement process. Code of conduct with commitment on confidentiality, nondisclosure of the e-procurement data content, incomplete and unauthorized data should be enforced in the system to ensure the accountability in the system and also outside the e-procurement system.

2.2 Accountability is misunderstood requiring to maintain only by the public officers.

Needs a communication and awareness program to make all the actors (i.e., public officers, regulatory/oversight agencies, bidders and civil society) in the procurement process and outside realize their accountability.

2.3 User ID and passwords are used in public procurement entities for carrying out the transactions on behalf of the authorized officers.

Needs a change management program followed by a comprehensive business process reengineering to streamline the processes and practices, and also capacitate the users on the new environment updating their perception, knowledge, attitude, leveraging the potential of e-procurement in a real sense.

190

Step no. Good governance aspect observation Recommendation

2.4 Knowledge is lost and accountability is diluted because of using personal e-mail accounts, obtained from free online e-mail services (i.e., e-mail accounts from Gmail, Yahoo, Microsoft, and other known and unknown service providers) in procurement transactions and communications in e-procurement system. The e-procurement system sends automated e-mails and transaction-related information to the e-mail account submitted at the time of their registration. When employees are transferred to another organization or discontinue their employment, or have bad relationship with the authority of the organization, the information and communication carried out through the use of personal e-mails are lost and knowledge and accountability not retained in the organization. It is not traceable.

Enforce an e-mail policy of using only official e-mail account either organization domain name based e-mail account issued by the corresponding organization or, the account issued by the government for its employees, or the e-mail account offered by the e-procurement system. This will help to recover the information, and also reuse the same e-mail account by new employee just changing user password.

2.5 The workflow process is there in the e-procurement system but it should be linked with the threshold given for the approving authority, and linked with given mandates or roles. The authority given to the public procurement officers for making decisions on varied and complex procurement is not adequate and also not linked in e-procurement system with their roles or mandates.

The public entity officials could be made more accountable only by giving appropriate and adequate authority, defining timeline for actions, thresholds of their authority, rewards for accomplishments and punishment or room for improvement, which could be built in to the e-procurement system.

2.6 Contract management module is not implemented yet.

Contract management module should be developed and implemented to make all actors accountable for ensuring the quality of deliveries within the agreed time and cost with the highest level of ethical behavior and attitude in executing the contracts.

2.7 It appeared that many of the attributes associated with accountability were attributed to public procurement entities and officers, whereas all the actors involved in the procurement process are accountable.

Clear communication program for all actors of the procurement process should be executed to strengthen perception, knowledge and attitude towards accountability and its nature and scope.

191

Step no. Good governance aspect observation Recommendation

3 Corruption Control 3.1 Face-to-face interaction at the time of

original hard copy submission of bid security, hard copy submission of online bids, requirement of physical participation of bidders in prebid meeting and bid opening (optional) is still encouraging corruption.

Adequate and effective business process reengineering is required to be carried out, and develop fully online mechanisms to eliminate face-to-face interactions between bidders and the public procuring entities before contract awards. Bid security could be submitted through the banks developing a module for the banks, providing functions to receive payments for registration of bidders, purchasing bid documents, issuing and managing bid securities and other guarantees, and also making payments to the bidders. Prebid meetings can be carried out using online interactive discussion forums in private mode. Bid opening can be done automatically by the e-procurement system, or by the authorized users of public procuring entities and subsequently share the bid opening records to all the bidders.

3.2 There is no facility to monitor and control the contract execution in e-procurement system.

The e-procurement system can provide greater control by developing a comprehensive e-contract management system, linking with the quality control, payments and expenditure tracking mechanisms.

4 Efficiency, Effectiveness, and Predictability 4.1 E-procurement system does not have

evaluation and contract management module.

Areas of efficiency gain, and effectiveness of the using taxpayer money could be achieved only through proper execution of the contracts, so contract management is important. Evaluation should be semiautomated.

4.2 Authority given to the public entity users is not adequate, and roles should be clear.

Workflow also should be linked with the roles and authority.

4.3 Compliance validation is limited. All the compliance requirements of the Public Procurement Act, Public Procurement Regulations, directives, outcome of reengineering should be incorporated in the e-procurement system.

4.4 There is a lack of knowledge of all the important features of the e-procurement system.

Training programs should be designed to provide the users training on getting real benefit and showing ways how the procurement can be done efficiently, effectively, and how to get benefit from the predictability features.

4.5 Reuse facility of bidding documents and bids are not known.

Bidding documents for public procuring entities, and old bid document for bidders should be possible to reuse with minor corrections in similar contracts.

5 Easy Access 5.1 Browsing speed is slow. Upgrade of datacenter with high-end servers, and

network equipment with failover arrangement and higher Internet bandwidth is recommended.

192

Step no. Good governance aspect observation Recommendation

5.2 There are no guidelines on using e-procurement system and legal validity of the transaction and records in the e-procurement system.

E-government procurement guidelines outlining the use of e-procurement system, and ensuring the legal authenticity of the transactions and records should be developed and issued as soon as possible.

5.3 Bidders getting support from the public procuring entities at the time of bid submission was observed.

This is conflict of interest, and should be stopped. Bidders should be trained for changes in their perception, knowledge, and attitude in new electronic environment transforming from the traditional one. Clear change management program should be developed and implemented.

5.4 Weak knowledge about complaint lodgment and its process.

The e-procurement system should provide easy access to the process of complaint lodgment, and also access to the review decisions.

6 Rule of Law and Equity 6.1 There are ambiguities between

traditional procurement process and e-procurement process in different aspects.

E-government procurement guidelines should be brought out as soon as possible to eliminate such ambiguities.

6.2 Many bidders do not know about e-procurement system and the process for participating in e-procurement.

Public procurement regulatory and monitoring agency is advised to start rigorous campaign to build awareness about the e-government procurement, and also bring in as many bidders as possible in the e-procurement system.

6.2 There are only limited number of bidders registered in the e-procurement system.

Only a critical mass of bidders registered in the e-procurement system, and competing in the procurements may bring value-for-money as well as promote the principle of equity in the procurements. In absence of adequate number of bidders in the e-procurement system, it will only institutionalize the organized syndication, cartel, and other malpractices.

7. Civil Society Awareness 7.1 There is no platform for open

discussion on the procurement issues. It is recommended to include an open discussion forum for the general public to participate and discuss on the procurement-related issues.

7.2 There is no clear policy of citizen as watchdog in public procurement.

Public procurement regulatory and monitoring agency is advised to work on the policy for citizen engagement in public procurement monitoring.

7.3 There is no helpdesk available in the e-procurement system.

Public procurement regulatory and monitoring agency is advised to provide helpdesk in its next version of e-procurement system with the helpdesk so that general public as well as all users can get quick support and response in relation to procurements.

7.4 There is no access provided to oversight agencies.

Public procurement regulatory and monitoring agency is advised to provide ad hoc access to audit trail and other forensic information for the investigation of fraud and corruption cases.

193

APPENDIX E. E-GOVERNMENT PROCUREMENT IMPLEMENTATION FRAMEWORK

Most of the governments around the world are undertaking the initiations to

implement e-government procurement (e-GP) systems as a tool for public procurement

reform for enhancing good governance in the public procurement sector. The introduction

of the e-GP system is expected to enhance the good governance, achieve better value-for-

money in the public procurement sector and support the socioeconomic development of

the country. Also as a result of the implementation of the e-GP system is expected to

improve transparency in the procurement process, ensure accountability, rule of law,

equity and competitiveness, improve corruption control, and increase civil society

awareness in regards to the public procurement. The implementation of e-procurement

has been considered one of the most promising and feasible options for the governments

in rendering enhanced governance in the acquisition of goods, works and services for the

public sector. Success of the e-GP implementation depends on numerous interdependent

factors and processes, which are sometimes ignored or not considered by the

implementers during its planning and implementation process. Governments around the

world have faced numerous challenges during implementation and adaptation of the e-GP

system, because of not having comprehensive framework for e-GP implementation with

adequate guidance.

This framework intends to provide step-by-step guidance to the governments or

organizations preparing for the introduction of e-GP system for good governance or

wants to improve the implementation strategy for achieving better results from the e-GP

194

systems. The e-GP implementation framework includes a comprehensive set of steps to

be taken and interventions to be considered for the introduction, implementation and

adaption of the e-GP system. The framework covers the challenges to be addressed at the

political level, administrative level, private sector and public community levels, as well

as legal, technical, and institutional level.

The e-GP implementation framework developed as part of the research is based

on the best practices and lessons learned from the experience, challenges, and solutions

adapted by many countries in implementing the e-GP system. The proposed e-GP

framework can be customized based on the country context in line with the focus of the

countries, their readiness, priorities, and strengths.

The e-GP implementation framework is divided into three broader phases (see

Figure E1): (a) Consensus Building, (b) Foundation Building, and (c) Implementation.

Recommended steps for the implementation of e-GP system is provided in Table E1 at

the end of this appendix.

Many indicators may inform a need of public procurement reform in the country.

country procurement assessment report, country integrated fiduciary assessment of public

financial management and procurement using the public expenditure and financial

accountability performance measurement framework of performance indicators and the

Organization for Economic Co-operation and Development Assistance Committee

country procurement assessment framework of baseline and compliance indicators and

other assessments carried out at different stages by the World Bank or other agencies may

inform such reforms.

195

Figure E1. E-government procurement implementation framework.

Step 1: Situational Analysis and Consensus Building

Step 1.1: Leadership/Sponsor

The first step in the country’s or organization’s implementation strategy is to find

a top-level leader or a senior-level person in the country or in each agency who could see

the long-term benefits and champion the move to e-procurement. It is important to make

a business case and convincing that first level why this is a good thing and why this is

going to help the state and their agency. Once there is a political and/or executive leader,

that leader plays a key role in marshaling the rest of the forces. If that leadership is not on

board, the whole initiative may be uncontrollable or, at the least,  very difficult. So the

196

first thing in initiating the e-GP would be identifying who that key sponsor was going to

be and working hand-in-hand for the initiative.

Step 1.2: Situational Assessments

To assess the current procurement status, it is recommended to carry out a

procurement performance baseline study for setting benchmark of the current

procurement performance of the public procuring entities, and regulatory/oversight

agencies as well as bidding community.

Step 1.3: E-GP Readiness Assessment

Based on the information from the baseline study, an e-GP readiness and

feasibility assessment should be carried out enhancing the assessment tool developed by

the Multilateral Development Bank for assessing the readiness of countries under nine

areas (see Figure E2) for the introduction of e-GP. The e-GP self-assessment

questionnaire should be sent to the key ministries, public procurement

regulatory/oversight agencies, procuring entities covering urban and regional areas,

professional and civil societies, information technology and telecom authorities, mobile

service providers, private sector businesses, the bidding community, and public

enterprises.

E-GP readiness assessment provides the status of the country against nine key

dimensions essential for the introduction of e-GP in the country. The results of the

benchmarking and e-GP readiness assessments should indicate that the country has good

levels of preparedness or supporting programs for such readiness.

197

Figure E2. E-government procurement readiness assessment.

Step 1.4: E-GP Strategy and Roadmap

E-procurement strategy should envision value-for-money and establishing good

governance in public procurement implementing the e-procurement system throughout

the country with a set of objectives against which the e-procurement program can be

monitored. The objectives could be summarized as three broad and high-level objectives

as shown in Figure 1, namely (a) improved governance through enhanced transparency

and accountability, (b) effectiveness through management information and efficient

processes, and (c) economic development through competitiveness and improved

investment climate. Key performance indicators should been identified under each of the

strategic objectives and targets should be set for the period of time for monitoring the

progress.

198

The e-GP strategy also should address policy issues in relation to legal, technical,

administrative, capacity building, governance model, sustainability model, human

resources model, authentication, third-party services (i.e., e-payment, Short Message

Service alert, etc.), operations and maintenance models, and a roadmap with clear action

plan for the implementation. A relevant competitive authority in the country should

endorse the e-GP strategy with full ownership and commitment for the implementation of

e-GP.

Step 1.5: Sensitization

Public awareness on e-GP and its benefits, challenges, purpose of

implementation, role of stakeholders should be communicated to general public, and

especially the political level, top-level administrator, policy makers, and civil societies

for the maximum utilization and sustainability of the e-GP system. This activity can be

implemented through organizing sensitization programs like workshops, seminars, audio-

video programs in radio and TV, and in paper media.

Step 2: Foundation Building

Development and adoption of a nationwide e-procurement system is one of the

key public sector management reform initiatives of a government. Step 2 activities will

prepare groundwork for the implementation of e-procurement system preparing itself

with the different preparatory tools, documents and capacities. Foundation building

activities include business process reengineering of the procurement processes and

practices, development of e-GP guidelines, development of business, functional, and

technical specifications of the e-GP system.

199

Step 2.1: Securing Budget

Securing the funding to ensure implementation happens as scheduled plan instead

of piecemeal is an important element of the preparatory process of e-GP system

implementation. It is important to secure required budget upfront to cover all the

activities. The budget should cover not only the software and hardware but also operation

and maintenance costs as well as change management and capacity-building costs.

Step 2.2: Business Process Reengineering

Mere automation of the paper-based procurement system does not give desired

efficiency, and leverage the available technology. It is important to streamline the manual

process and practices by business process reengineering first, which help improve the

efficiency gains benefits because of the streamlined procurement process. Current

procurement processes and practices should be reengineered to use modern technology.

This will require changing the way the procuring entities prepare the bidding documents,

bidders prepare the bids, fees are collected for the bidding dossier purchase, bid securities

are handled, processes evaluated and communication with the bidders and other

stakeholders is managed, contracts are managed, and complaints are handled.

Simplified/reengineered procurement processes and practices will allow the

stakeholders to operate in a transparent, efficient, cost-effective, and timely manner and

reduce administrative burdens on procuring entities and bidding communities leveraging

the e-GP.

Step 2.3: E-GP Guidelines

Transition from a paper-based procurement system to an electronic system entails

significant reengineering in process, practices, and means as well as brings up a new

200

relationship with existing and new stakeholders. When most of the requirements of

compliance with the procurement acts and regulations are automatically addressed by the

e-GP system, new areas of performance-based roles of agencies are emerged.

Automation of processes and practices under the e-GP will generate new

relationships between stakeholders—regulatory/oversight agencies, bidding community,

banks, and with others, and new roles to be performed by different agencies. For

example, the e-GP system would have to be linked with different government e-services

for accessing, verifying, and validating the authenticity of the data to be used in e-GP. In

addition, the different laws and regulations governing electronic transactions (e.g.,

information and communications technologies, e-transaction, and e-signature acts) as

well as national e-government strategies, e-procurement strategies and other master

guidelines, would have to be synchronized and harmonized with the Public Procurement

Act and Public Procurement Regulations.

For making the implementation of the e-GP system well-accepted and free of

ambiguities while the majority of procuring entities are still following the paper-based

procurement practices, introducing the e-GP system should be supported by the

introduction of e-GP guidelines adequately endorsed by relevant competent authority of

the country. In addition to the e-GP guidelines, the standard bidding documents also to be

reengineered making friendly with the e-GP system requirements. This requires that e-GP

guidelines to govern the operation, use, performance monitoring by defining procurement

performance monitoring framework, and scope of the e-GP system in the country. The e-

GP guidelines will serve as the governing principles

Step 2.4: Development of E-GP Business,

201

Functional and Technical Specifications

The government initiative towards the introduction of a unitary e-GP in the

country can generate significant efficiency gains in public procurement sector. The e-GP

system improves transparency, accountability, procurement management, strengthens

documentation, compliance and performance audit capabilities around the procurement

function and improving good governance in public procurement sector.

The e-GP specification should be based on the prevailing acts, regulations,

practices, and e-GP guidelines and at the same time taking leverage of the information

and communications technologies to make the public procurement transparent, efficient,

effective, and cover the end-to-end procurement processes for all procurement methods

with adequate reengineering. It should clearly indicate the business, functional, and

technical requirements of e-GP system providing functional facilities to all the major

users.

It should define the complete scope of the e-GP system and then define the

deliverables by the software supplier and prepare the bidding document for national or

international competitive bidding for the acquisition of an e-GP system.

Major functionalities to be supported by the e-GP system are listed as follows:

• Centralized registration system for procuring entities, and all other users of e-procurement system

• User-specific administrative dashboards

• E-bidding (covering all processes and procurement methods for goods, works, and services supported by public procurement law and regulations of the country from demand aggregation, annual procurement plan preparation, e-advertise, e-submission, e-evaluation, e-notice to signing of contract)

202

• E-catalog

• Simplified bidding (e-quoting)

• E-framework agreements and their management

• E-auctions (forward and reverse auction)

• E-contract management system

• E-invoicing, payment, and guarantee (through banks and other options)

• Complaint and dispute handling

• E-procurement performance measurement and monitoring system

• Interface with Integrated Financial Management System, company registrar’s office, banks, tax, and other e-services relevant to e-GP

• User activities under administrative workflow and system audit trails

• Online help

Functionalities could be more or less based on the country context.

Step 3: Implementation and Adaptation

Step 3.1: Acquisition of E-GP System

In implementation, acquiring e-GP system is pivotal. There are different options

for acquiring the e-GP application:

• Off-the-shelf e-GP application with customization (recommended)

• Custom-tailored

• Software-As-A-Service—Cloud-based, future trend

• Government-to-government support

Acquisition of the e-GP system is recommended to carry out through an

international competitive bidding inviting the bidders having already developed e-GP

203

system with required configuration and customization to map with the country’s e-GP

requirements. There are many off-the-shelf e-GP applications available in the market,

which can be used with minimum configuration of parameters, and changes. The e-GP

system supplier will be required to develop the functionalities if any requirements

specific to the country procurement environment are not available in offered existing

system. The e-GP system should be procured with the perpetual license and license to

modify and update the source code by the client as required, and to be used across the

country in all the procuring entities, public entities, all other users including national and

International bidders.

Step 3.2: Data Center Preparation

Preparation of data center includes the assistance for building data center.

Building the data center involves installation of power solutions and power backup,

precision air-conditioning, fire alarm system, digital closed-circuit television, surge

suppressor, access control system, water detection system, dehumidifier, pest repellent,

data rack, raised flooring, electrical and civil works, and passive components like

network cables, patches, and so on, procurement of data center and mirror center servers,

network equipment, storage, power solution, Internet connectivity, and required system

software and licenses

Step 3.3: Capacity Building and Change Management

Change management is one of the important activities to be carried out at all

levels of user environments for the successful adaptation of the e-GP system. For public

procuring entities, the new rules and practices in the electronic environment will require

changes to the way they work, for example, in use of new procedures, as well as the

204

technology mandates. For some, the appropriate technology will already be in place. For

others, that are operating public procurement in a purely manual manner, there is a need

to take necessary change actions in short time to ensure e-GP system can be properly

implemented. Change management could be a mammoth, but sometimes you can

piggyback the already started initiatives. Sometimes, if the country already have

implemented any e-services, like IFMIS, the trails made by those implementations can be

followed. For the change management, changes to be managed in the following areas:

• Structural/institutional

• New relationship management with new and old stakeholders

• Administrative processes and practices

• Emotional change management—Emotional readiness is very important for success. • Developing leadership at political, social, different levels of government

bureaucracy, private sector bidding community

• Managing resistance in public procuring entities and bidding community

• Devolved power management

• Capacity enhancement and building For capacity building, the following activities could be carried out:

• Different levels of training program should be conducted to address the capacity needs of different users involved in the procurement processes (i.e., training programs for procuring entity procurement users, bidders, evaluation committee members, accounting officers, bank users, oversight agency users, etc.). Training programs will be in the form of regular courses as well as ad hoc trainings on demand basis. Training programs should be conducted at the all-administrative levels of public procurement.

• Benchmark study visits for the selected key users should be conducted in few of the developed and developing countries for gaining knowledge and learn from the experience on the implementation of e-GP system.

205

• For the effective adaptation of the e-GP system by its key users and makethe e-GP system successful, initial support on its operation, maintenance,implementation management is very important. For this, it is important toestablish helpdesk, resource with project management team and resourcesfor e-GP and data center operation and maintenance.

3.4: Pilot and Full Rollout Across All Public Procuring Entities

Implementation of e-GP system is recommended to be adopted in phased rollout

model as opposed to the traditional phased module activation model. This means that the

initial system developed shall cover all methods of procurement and stages of the

procurement process and all the required functionality to enable piloting of a complete

system. Procuring entities will chose which procurement method to start with based on

their availability of procurement packages under the annual procurement plan.

Phasing should be standalone. E-procurement processes should be complete for

any procurements. Options of parallel submission of hardcopy bids and also online

submission for the same procurement should not be encouraged. It options provided, it

will only delay the adaptation of the e-GP system and support the procurement reform in

true sense. All the procurement methods should be developed from the very beginning.

Otherwise with this what happens is that even a department using e-procurement will

have to use e-procurement for one of the procurement and manual system for others. This

kind of e-procurement implementation does not give desired benefits. It also does not

improve efficiency of the agency but adds more burden for capacity requirements as well

as resources for both types of procurements. It will only promote systemic corruption.

Phases of rollout would be divided into two phases:

1. Pilot in selected agencies

206

2. Wider rollout throughout the country

The e-procurement system implementation should be in phased manner selecting

few key public procuring entities, which provides an opportunity to reduce the e-

procurement system failure risks, test the functionalities and processes in e-procurement

system, and extend the demonstration effect in rolling out the system in other procuring

entities based on the success during pilot implementation. It helps enhance the rollout

plan based on the lessons learned during pilot implementation.

Success of the implementation of e-GP depends on many factors. Some of the

major factors are the readiness of the agencies in terms of willingness and commitment of

leadership in the organization to take up e-procurement, the level of use of information

and communications technologies by the procurement-process-related personnel, and

diversity of procurements carried out by those agencies.

Number of pilot agencies to be nominated depends on the country context. It

could be four to 10 pilot agencies, considering their willingness and commitment to drive

e-procurement initiative and information and communications technologies readiness:

• One or two representative pilot agencies each could be chosen from among the agencies focusing on goods, works and services procurement

• Procuring entities with specific nature of procurements (i.e., medical, information and communications technologies, rations, etc.)

• Procuring agencies already doing some forms procurement automation

• Procuring agencies representing geographical reach and including at least two local governments (only agencies passing the readiness requirements)

• The procurement regulatory agency of the country as the administrator of the e-procurement system

207

Wider rollout implementation will be carried out based on further assessment of

the procuring entities on their readiness.

Rollout strategy also could be carried out based on (a) stages of procurement

processes, (b) procurement method, (c) threshold, (d) readiness, and (e) strategic

government priority sector.

3.5: Operations and Management

Regular system maintenance, updating e-GP portal content, updating procurement

rules and processes, supporting e-GP users, and ensuring availability of e-GP system to

its users all the time should be addressed through a well-designed operation and

management policy. The operation and management arrangement should be made in such

a way that the e-GP system is up and running all the time 24/7/365 for procurement

transactions, all users are supported by a helpdesk, and the e-GP portal remains always

updated with its content and processes.

3.6: Third-Party E-GP System Audit

When the e-GP system is successfully tested and stable after pilot implementation

and rollout started in other public entities, an independent audit agency is required to

verify compliance of the e-GP system to the Public Procurement Act and Public

Procurement Regulations of the country and a set of assessment guidelines drawn from

international best practices, and also actively guide and, as required, provide hands-on

support to enable the implementation comply with the stated best practices. Third-party

audit provides confidence to the stakeholders using the e-GP system, and provides

authenticity of the transactions in the system. Third-party audit may contain the following

assessments:

208

Security audit. Evaluate security controls implemented in the application

software vis-à-vis international best practices and make recommendation on the remedial

measures to be taken to comply with the prescribed global standards. Besides the

application audit, the security audit shall review security controls in database and

operating systems.

Documentation. Prescribe documentation requirements in a typical information

technology software implementation project (i.e., both software development and

maintenance phases cover not only the software application but also the installation,

implementation, and exit/transition management activities) and actively guide the team in

getting the documentation done in line with international standards.

Server infrastructure deployment and data security. Evaluate the existing

deploymentarchitecture (server infrastructure) and actively guide the implementation

team in reconfiguration of the servers and storage such that load handling is distributed

across the servers and recovery point objective and recovery time objective are kept as

minimal as possible.Especially, server configuration and data security have to be

evaluated firsthand and hands-on guidance should be provided to the implementation

team in reconfiguration of the server infrastructure. A robust data backup policy has to be

effectively implemented.

Penetration testing. Perform external penetration testing to identify

vulnerabilities that could be exploited by users located outside the network perimeter and

provide hands-on guidance to the implementation team in strengthening perimeter

security.

209

Table E1. E-Government Procurement Implementation Framework Checklist

Step no. Activity Purpose Outcome

Estimated time

1 Situational analysis and consensus building 1.1 Identification of

leader/champion/ sponsor

Put forward e-government procurement (e-GP) initiative

• Leader, sponsor, orchampion identified

• Sponsor the idea ofe-GP system

1.2 Procurement performance baseline survey

Situational assessments 3 months

1.3 E-GP readiness assessment

Readiness assessment to identify the status of required preparedness for the implementation of e-GP system

E-GP readiness assessment report with the strengths, gaps, and perceptions

2 months

1.4 Development of e-GP strategy and roadmap

To ensure top-level political and government commitment and define the policy, strategy and step-by-step actions for the successful implementation of e-GP system

• National strategyfor e-GP system implementation with action plan

• Political andexecutivecommitment

3 months

1.5 Sensitization about e-GP

A broad sensitization and awareness program should be implemented with an objective to

• Raise awareness of the e-GPinitiative

• Get buy-in from stakeholders

• Evaluate and get feedback onthe initiatives

• Raise awareness of benefits ofe-GP

• Keep stakeholders informed asthe e-GP implementationprogresses

• Disseminate the outputs of thee-GP implementation

• Strong political,executive levelsupport andcommitment

• Aware civil society

• Adoption of changeby public procuringentities and biddingcommunity

210

Table E1. E-Government Procurement Implementation Framework Checklist (continued)

Step no. Activity Purpose Outcome

Estimated time

2 Foundation building 2.1 Securing budget for

the e-GP program Government commitment on e-GP implementation

Secured required budget for e-GP implementation

2.2 Carry out business process reengineering

Streamline traditional procurement processes and practices in line with electronic environment leveraging information and communications technologies for strengthening transparency, accountability, corruption control, efficiency, effectiveness, predictability, rule of law and equity, easy access to information, and civil society awareness

Reengineered and standard processes and practices for e-GP

3 months

2.3 Preparation of e-GP guidelines

To provide legal and operational authenticity of the e-GP, its underlying electronic processes and incorporated operational practices different from the traditional procurement approach, authenticity of the transactions done through e-GP by its users, ownership of data and content stored in the e-GP system, security and institutional arrangements for its governance, and roles, responsibility, and obligations of the users

E-GP guidelines or directive

3 months

2.4 Development of e-GP business, functional, and technical specifications

Outline business, functional and technical requirements to be covered by e-GP system, and its operating environment

Specification document of e-GP system for the acquisition of the system and related services, and its infrastructure

2 months

211

Table E1. E-Government Procurement Implementation Framework Checklist (continued)

Step no. Activity Purpose Outcome

Estimated time

3 Implementation and adaptation 3.1 Acquisition of e-GP

system Competitive bidding process to acquire most appropriate e-GP system for the implementation with design, configuration/ customization, and development, and commissioning

Installed operational e-GP system

Acquisition of e-GP system

3.2 Data center preparation

Preparation of data center addressing desired security and performance requirements for the e-GP system operations

Physical environment of data center with virtual platforms for the development, testing, training, and transactions (production)

3 months

3.3 Capacity building and change management

To train users to operate e-GP system and instill required change culture to operate in electronic environment leveraging information and communications technologies

Trained users Users with positive attitude towards change

3.4 Pilot and full rollout across all the public procuring entities

Select limited number of procuring agencies with full enthusiasm and readiness in terms of motivation, capacity, and information and communications technologies infrastructure to use the innovative tool in public procurement Test the complete cycle of procurements using different procurement methods

Number of public entities test the procurement cycle in electronic environment Ensured stable and robust e-GP system Confidence development for other agencies for further nationwide rollout

One procure-ment cycle (3 months to 1 year)

3.5 Operations and management

Regular system maintenance, updating e-GP portal content, update procurement rules and processes, support e-GP users, ensure availability of e-GP system to its users

Live e-GP system all the time 24/7/365 for procurement transactions Happy users getting required support for its operations Updated procurement information and processes

Ongoing during the life of e-GP

3.6 Third-party e-GP system audit

Ensure security, confidentiality, and high level of operational quality of e-GP system and its management arrangements

Validated and trusted e-GP system for the use in public procurement

3 months