Copyright 2013 Sharmila Murthy, Laura Pereira, Alicia Harley, Daniel Shemie, Eunjee Lee, Patricia Guardabassi, Chao Zhang, Scott Moore, and the President and Fellows of Harvard

College

The Role of Information and Communications Technology (ICT) in Helping Decision-Makers Meet Food, Energy and Water (FEW) Needs

Sharmila Murthy, Laura Pereira, Alicia Harley, Daniel Shemie, Eunjee Lee, Patricia Guardabassi, Chao Zhang, and Scott Moore

Sustainability Science Program

Working Paper 2013-02

November 2013

The Role of Information and Communications Technology (ICT) in Helping Decision-Makers Meet Food, Energy and Water (FEW) Needs Sharmila Murthy, Laura Pereira, Alicia Harley, Daniel Shemie, Eunjee Lee, Patricia Guardabassi, Chao Zhang, and Scott Moore Abstract This report outlines the key themes that emerged at a one-day inter-disciplinary workshop held at the Harvard Kennedy School of Government on Saturday, May 18, 2013, which focused on the question:

What role can the data generated through Information and Communications Technology (ICT) play in aiding decision-making to meet current and future food, energy and water (FEW) needs in the wake of climate change?

The goal of this workshop was to help define an inter-disciplinary, scholarly research agenda to help address this critical question. Its scope was purposefully broad and reflected an attempt to bridge divides across academic disciplines and to foster conversation between technologists, policymakers and academics. The workshop was focused on exploring the current and potential uses of Information and Communication Technologies (ICTs) in the food, energy and water (FEW) sectors and also drawing lessons from the use of ICT in the broader field of development. However, it emerged that the use of ICT for development (“ICTD”) was still evolving and that many of the challenges to the effective deployment of ICT for FEW had parallels in other sectors.

Our hypothesis was that useful technologies already exist, but critical policy barriers prevent their effective use and deployment. Accordingly, the starting premise of the workshop was that while ICT is not a silver bullet, it has the potential to: (i) improve decision-making on complex problems at all scales, and (ii) link data across multiple levels for more thorough analysis. However, despite the original focus on the FEW nexus, the actual discussion focused much more broadly on the role ICT has already played in development. Participants shared experiences from a wide range of development sectors, including health, post-conflict, gender and sanitation, in addition to food, energy, water and climate change. Section II of this report provides further background on the scope and goals of the workshop.

Five key areas emerged from the workshop. These include i. the role ICT can play in linking global governance with local knowledge and preferences, ii. the ethical dilemmas that have arisen in the ICTD context, iii. how ICTD can inform policy-making across different scales, and iv. the importance of context in employing ICT tools.

Key research questions for further study also emerged from the workshop. These fell under six broad themes of

v. addressing local needs and preferences effectively, vi. the kind of decision support tools ICT can provide to policymakers, vii. how “small infrastructure” (mobiles) relies on “large infrastructure” (communication towers), viii. the appropriateness of the market for selecting ICT tools, ix. why more data have not necessarily led to more effective decision-making, and x. important questions about privacy and data misuse.

Overall the workshop was judged a great success by all and we hope that it has established fertile ground for future trans-disciplinary research to be conducted in this important space. Keywords: Information and Communications Technology (ICT), Development (ICT4D), Food-Energy-Water Security, Policy

Citation This paper may be cited as: Murthy, S., L. Pereira, A. Harley, D. Shemie, D., E. Lee, P. Guardabassi, C. Zhang, and S. Moore. 2013. The Role of Information and Communications Technology (ICT) in Helping Decision-Makers Meet Food, Energy and Water (FEW) Needs. Sustainability Science Program Working Paper 2013-02. Sustainability Science Program, Kennedy School of Government, Harvard University, Cambridge, MA. It is available at http://www.hks.harvard.edu/centers/mrcbg/programs/sustsci/documents/papers/2013-02. Prof. William Clark has approved this paper for inclusion in the working paper series. Comments are welcome and may be directed to the author, [sharmila.murthy@hks.harvard.edu]. Author Biographies

Sharmila Murthy, Sustainability Science Fellow and Carr Center for Human Rights Policy Fellow (2012–2013), Harvard Kennedy School. (Co-Lead Workshop Organizer) Laura Pereira, Sustainability Science Fellow (2012–2013), Harvard Kennedy School. (Co-Lead Workshop Organizer) Alicia Harley, PhD candidate and Sustainability Science Fellow, Harvard Kennedy School. (Workshop Co-Organizer) Daniel Shemie, Co-Founder and project manager at mWater and former organizer of water-related ‘hackathons’ at World Bank. Eunjee Lee, Sustainability Science Fellow (2012–2014), Harvard Kennedy School. (Workshop Co-Organizer) Patricia Guardabassi, Sustainability Science Fellow (2012–2013), Harvard Kennedy School. (Workshop Co-Lead Organizer) Chao Zhang, Sustainability Science Fellow (2012-–2013), Harvard Kennedy School. (Workshop Co- Organizer) Scott Moore, Sustainability Science Fellow (2012–2014), Harvard Kennedy School. (Workshop Co-Organizer)

Sustainability Science Program The Sustainability Science Program at Harvard University harnesses the University’s strengths to promote the design of institutions, policies and practices that support sustainable development. The Program addresses the challenge of sustainable development by • advancing scientific understanding of human-environment systems; • improving linkages between research and policy communities; and • building capacity for linking knowledge with action to promote sustainability. The Program supports major initiatives in policy-relevant research, faculty research, training of students and fellows, teaching and outreach. See http://www.hks.harvard.edu/centers/mrcbg/programs/sustsci.

Author Acknowledgements This work was conducted while the authors were Fellows in the Sustainability Science Program at Harvard University. The workshop was made possible through funds from the Sustainability Science Program. Support from the Italian Ministry for the Environment, Land and Sea and the Mossavar-Rahmani Center for Business and Government is gratefully acknowledged. Special thanks also goes to Professor William Clark and Nancy Dickson for their guidance and support of the project and to all of the workshop participants (listed in Appendix A) who generously contributed their time and energy to the workshop. The views expressed in this paper are those of the author and do not necessarily reflect those of the Sustainability Science Program, of the Mossavar-Rahmani Center for Business and Government, or of Harvard University. The Sustainability Science Program Working Papers have not undergone formal review and approval. Such papers are included in this series to elicit feedback and to encourage debate on important public policy challenges. Copyright belongs to the author. Papers may be downloaded for personal use only.

Table of Contents I. Executive Summary ................................................................................................................. 1 II. Background and Goals of the Workshop ............................................................................. 3 III. ICT for Development: Considering the Local Context ........................................................ 5

A. How Can ICT Be Culturally Appropriate? ...................................................................... 6

B. What Are the Incentives to Engage? ................................................................................ 7

C. Is Technology Socially Neutral or Can It Exacerbate Inequalities? ................................ 8

D. To What Degree Does “Small Infrastructure” (ICT) Depend on “Big Infrastructure”? .. 9

E. How Can Culturally Appropriate Technology Be Developed? ..................................... 11

IV. What Are the Ethical Implications of the Rise of ICTD? .................................................. 12 V. ICT Across Scales: How Does ICTD Inform Policy-Making? ......................................... 13

A. The Advent of “Big Data” .............................................................................................. 13

B. Are All Data Equal? The Challenges of Accuracy, Verification and Analysis ............ 15

C. Does More Data Lead to Better Decision-making? ....................................................... 16

D. Can ICT Enable the Forecasting of Future Events? ....................................................... 17

VI. Framing the Big Picture: What Is the Role of Theory? ..................................................... 18 A. What Role Do Institutions Play in Promoting ICTD?.................................................... 18

B. What Is the Role of Theory and How Does It Inform Practice? .................................... 19

C. Is There a Need for Decision-support Tools? ................................................................ 21

VII. Conclusion ......................................................................................................................... 21 VIII. Appendices ..................................................................................................................... 23

A. List of Participants and Group Photo ............................................................................. 23

B. ICT for FEW Workshop: Original Agenda .................................................................... 25

C. List of Questions Used to Form the Break-out Groups .................................................. 26

IX. References .......................................................................................................................... 29

1

I. Executive Summary This report outlines the key themes that emerged at a one-day inter-disciplinary workshop held at the Harvard Kennedy School of Government on Saturday, May 18, 2013, which focused on the question:

What role can the data generated through Information and Communications Technology (ICT) play in aiding decision-making to meet current and future food, energy and water (FEW) needs in the wake of climate change?

The goal of this workshop was to help define an inter-disciplinary, scholarly research agenda that can help to address this critical question. Accordingly, the workshop was focused on exploring the current potential uses of Information and Communication Technologies (ICTs) in the food, energy and water (FEW) sectors and also drawing lessons from the use of ICT in the broader field of development. However, it emerged that the use of ICT for development (“ICTD”) was still evolving and that many of the challenges to the effective deployment of ICT for FEW had parallels in other sectors. The workshop was organized by an inter-disciplinary group of Fellows in the Sustainability Science Program at Harvard Kennedy School during the 2012-2013 academic year and included experts from the broader field of ICT for development (see Appendix A for the list of participants). The format of the workshop consisted of brief presentations followed by moderated open discussion (see Appendix B for the agenda). Since a lot of attention is traditionally paid to the “potential of ICT,” presenters were encouraged to address concrete examples and the factors that led to effective implementation. They were also urged to focus on barriers to implementation and unsuccessful ICT projects. The conference did not treat ICT as a panacea or ‘silver bullet’ but instead encouraged a frank and honest discussion about the pitfalls and gaps that researchers need to be aware of when using ICT. Participants were also invited to write on ‘sticky notes’ key issues and questions that they wanted to discuss—and these questions formed the basis of the topics of the break-out groups (see Appendix C). Our hypothesis was that useful technologies already exist, but critical policy barriers prevent their effective use and deployment. Accordingly, the starting premise of the workshop was that while ICT is not a silver bullet, it has the potential to: (i) improve decision-making on complex problems at all scales, and (ii) link data across multiple levels for more thorough analysis. The scope of the workshop was purposefully broad and reflected an attempt to bridge divides across academic disciplines and to foster conversation between technologists, policymakers and academics. Despite the original focus on the FEW nexus, the actual discussion focused much more broadly on the role ICT has already played in development. Participants shared experiences from a wide range of development sectors, including health, post-conflict, gender and sanitation, in addition to food, energy, water and climate change. Section II of this report provides further background on the scope and goals of the workshop. The workshop explored the role ICT can play in linking global governance with local knowledge and preferences. Major themes that emerged included

1. the need to develop culturally appropriate strategies for ICT implementation; 2. incentives to participate in ICT programs and projects by end users and other actors; 3. the recognition that technology is not socially neutral and can exacerbate inequalities; 4. the extent to which “small” infrastructure depends on “large” infrastructure; and 5. processes for and barriers to developing appropriate ICT technologies.

2

These themes are discussed in Section III below. The workshop also delved into some of the ethical dilemmas that have arisen in the ICTD context, which are examined in Section IV of this report. These included discussions on

1. how ICT has changed the flow of information and how research is conducted, potentially altering the ethical obligations of those collecting data;

2. the reliability of data with an increased amount of data flowing horizontally between people and not just vertically; and

3. the need to revisit privacy, confidentiality and security as more people share data and it is easier to collect and access.

Participants also discussed how ICTD informed policy-making across different scales. The resulting themes included

1. the advent of “big data”; 2. the challenges of accuracy and verification; 3. the extent to which more data leads to better decision-making; and 4. the continued limitation of data analysis to predict future vulnerabilities.

These are discussed in Section V below. Finally, the participants considered the importance of context. In break-out groups, the participants discussed the following issues in more depth:

1. the role that institutions, or “rules of the game,” (North 1990) play in determining successful projects;

2. how theory can meet practice, including the role of ICT as a tool for “boundary work” (Cash et al. 2003) for linking knowledge with action, enabling participants to contribute to shared problem definitions and goals; and

3. processes for promoting generalizable guidelines and decision-making tools for the design of appropriate ICT technology in different contexts and for different goals.

These topics are considered in Section VI. By bringing together technologists, policymakers and scholars, the workshop sought to identify critical areas for future research. Key research questions that emerged for further study include:

I. It is almost axiomatic to say that ICT for FEW strategies need to be demand-driven, culturally appropriate and take into account local needs and preferences. What are the reasons that this may not happen, and what factors can allow for local needs to be addressed effectively?

i. Is it possible to meet the needs of policymakers and local communities with the same strategy? Can ICT be utilized to understand and potentially align the needs of multiple stakeholders?

II. What kind of decision support tools can ICT provide to policymakers to ensure that FEW strategies are implemented effectively?

i. What factors may cause ICT for FEW strategies to (perhaps inadvertently) exacerbate existing inequalities in a community and entrench local power dynamics? How can this be avoided?

ii. Does an ICT strategy differ if it seeks to enhance “vertical” communication (i.e., gathering local data to inform policymaking) as opposed to ‘horizontal’ communication (i.e., people sharing data among themselves)?

III. To what degree does “small” infrastructure, like mobile technology, depend on “large” infrastructure, like power lines and internet cables?

i. Is there a danger that excitement over the potential role of ICT may distract from the need for investments in more mundane infrastructure?

3

ii. Or, does the increased participation and transparency that can come from an ICT strategy create the political will needed to pressure improved access to basic infrastructure?

IV. Does the market currently do an effective job at selecting for effective ICT for FEW strategies and weeding out those that are doomed to fail?

i. Is there a need for another mechanism, such as an independent verification body? ii. How are data from ICT currently evaluated?

V. More data has not necessarily led to more effective decision-making, especially in the FEW sectors. Why is this?

i. What factors create the political will needed to ensure that ICT for FEW strategies actually leads to improved evidence-based decision-making, transparency and/or cooperation over transboundary resources?

ii. Is this entirely dependent on the local/national political situation, or can the ICT strategy itself play a role?

iii. Although “data” is popular, analysis appears not to be. What interventions in our analysis of ICT data aid in improving transparent decision-making?

VI. Data sharing raises important questions regarding privacy and misuse of data. i. What kind of ethical guidelines need to be established for ICTD? What institutions need

to be in place to ensure that both the collection of data as well as how they are shared is ethical?

ii. Can traditional development institutions offer models for what ethical norms should be?

II. Background and Goals of the Workshop Information and communications technology (ICT) has burgeoned throughout the world and continues to spread, often being viewed as a virtually indispensable tool to international development (Heeks 2008; Maumbe and Okello 2010; Unwin 2009). Many have seen the ICT revolution as an opportunity to harness the power of this innovation in order to meet the needs of the world’s poorest (Kalil 2009). Sachs, for example, has hailed mobile phones and wireless internet as “the most transformative technology of economic development of our time” (Sachs 2008). While the ‘upstream’ parts of the ICT for development value chain—infrastructure, accessibility and use—remain important, there is now evidence of ‘downstream’ impacts in the form of economic growth, sustainable livelihoods and greater freedom (Heeks 2010, 635). Research suggests that access to mobile phones play an increasingly important role in economic development (Donner and Escobari 2010; Aker and Mbiti 2010), but that such technology is not a ‘silver bullet’ for development (Aker and Mbiti 2010). There has been increasing focus on whether the internet empowers and creates mechanisms for improving public participation (Wheeler 2011; Pereira et al. 2003). At the same time, a strong emphasis on ICT-oriented development can reinforce and entrench existing power relations, such as between men and women, due to the unequal penetration rate of mobile services (Etzo and Collender 2010). This finding is underlined by two different studies from India that also highlight the importance of developing an appropriate program to ensure access by marginalized groups (Cecchini and Scott 2003; Kuriyan, Ray, and Toyama 2008). Finally, as Avgerou highlights, it is critical to understand the socio-political context in which ICT innovation takes place and to develop rigorous theory to avoid reliance on “common sense or popular assumptions of what are desirable development effects” (Avgerou 2010, 15). This workshop focused on the FEW nexus, a term that increasingly refers to a ‘perfect storm’ facing humanity: of food, water and energy crises compounded by a changing climate (Beddington 2009). Although important in their own right, these challenges are also interconnected and feedbacks from one impact the others, which adds to the complexity of decision-making around these needs. Given these challenges, it is crucial to understand what sort of information is most valuable across different

4

scales and levels in order to make long, medium and short-term decisions to meet these needs. In order to make strategic decisions about meeting FEW security requirements, historical data and future scenarios are vitally important. It is also necessary to try to appreciate the risks, trade-offs and feedbacks from investments in one sector on the other sectors and activities. For example, investing in agricultural production is a strategic decision that needs to be understood in light of alternative land uses as well as the social and environmental consequences of agricultural production. Farming has positive feedbacks not only into the climate system (through emissions and land use change), but into other socio-ecological systems, such as water, and is equally impacted by the energy sector through examples like the food-fuel debate around biofuels. FEW-related decision-making is often scattered among different actors (government, industry, multi-lateral institutions, etc.) and influenced by various formal and informal institutional frameworks. For instance, the municipal delivery of water and sanitation services is impacted by the way that water is allocated and managed as a resource, and also by laws that govern pollution and development. Moreover, sometimes development goals in one sector can create conflicts with sustainability goals in another. For example, a policy that seeks to intensify agricultural development in a region could conflict with a desire to improve water security and sustainability; or a policy designed to promote energy access through biofuels could inadvertently result in an increase in food prices or unsustainable agricultural practices. Thus, there is a need to look at data across levels to see whether there are potentially useful data synergies where information generated at one level can be useful at another level or in a different sector. Yet, often there is a trade-off between meeting current needs and planning more long-term projects that address an uncertain future.

This workshop sought to examine whether ICT could be better utilized to improve decision-making processes related to the FEW nexus. In particular, the workshop was interested in influencing policy-makers and other stakeholders tasked with reconciling the need to reduce communities’ current vulnerability to disruptions in their access to food, energy and water services with the need to invest in sustainability projects that offer long-term resilience to communities facing an uncertain future. Added to this mandate is the issue of limited resources to execute both of these necessary tasks. By improving access to current information as to the state of FEW needs in communities and then mapping these onto larger vulnerability maps given future scenarios of climate, economy etc., resources could be targeted in a way that improves both current and future access to these human rights. At the same time, involving communities in generating their own data aids in systemic learning within communities and provides (often marginalized) people with tools for actively engaging in governance (Corburn 2005; Ceccarelli and Grando 2007).

The workshop asked whether ICT could further the communication, translation and mediation that is essential to effectively bridge boundaries between science and policy, and also across information available at different scales (local to global). The importance of linking the local to the global is recognized in the field of Science, Technology and Society (Jasanoff and Martello 2004). Moreover, a number of studies have highlighted the “importance to effective science advising of ‘boundary work’ carried out at the interface between communities of experts and communities of decision makers,” which is critical to overcoming the communication gap that often exists between these two groups (Cash et al. 2003). The starting premise of the workshop was that, as researchers, we had identified a dearth of useful, accessible data to guide FEW policy across multiple levels, compounding and exacerbating the limited institutional capacity in many developing countries. In addition, recognizing that there was a growing body of literature that highlights the potential of ICT for development, but that also cautions against naïve optimism, the workshop organizers sought to understand better how ICT can be harnessed to inform decision-making and broader policymaking, especially with respect to food, energy, water and climate change issues.

5

The workshop agenda was originally driven by a series of questions:

1. How can data from ICT aid sustainable development through achieving FEW needs?

i. For example, in terms of a sustainability criterion (i.e., looking into the future), how can more abstract data generated for example from climate models, be translated down and become useful for decision-makers now?

ii. In terms of an equity criterion, how can ICT help to map areas where there is limited or no service delivery and then how can this knowledge be integrated into formal decision-making around resource allocation?

iii. Given the vast array of ICTs, which have the greatest capacity or potential for addressing FEW nexus challenges?

2. Can the revolution that data from ICT provides be leveraged to aid cross-level engagements between multiple stakeholders around meeting these needs?

i. How can we begin to bridge cross-level data that does not necessarily overlap, but that is vital to understand if we are to make decisions around meeting needs both equitably and sustainably?

3. What types of data are needed in order to create and implement policy for meeting FEW needs?

i. What are the barriers to providing these data and can improvements in ICT help to overcome these?

ii. Does the ability of ICT to provide “real time” information enable policy-makers to develop policies that are more flexible and adaptable, especially in light of changing climatic conditions?

4. Can ICT provide the means of overcoming infrastructural and institutional constraints in order to provide better information in a more efficient way to meet FEW needs?

i. Under what conditions and what are the major barriers to ICT for this purpose?

ii. Moreover, can ICT provide an opportunity for local stakeholders to provide feedback that can inform decision-making at a higher level?

With these questions in mind, the workshop was organized as set forth in the Agenda in Appendix B. As the goal was to foster cross-cutting conversation, formal presentations were brief and participants spent most of the time discussing and debate across the table and in small groups. The next sections summarize the key themes that emerged.

III. ICT for Development: Considering the Local Context The workshop participants discussed a broad range of ways in which ICT could be useful in a development context. A variety of different tools for collecting, analyzing and communicating information were discussed, including using cellphones, smartphones, automated sensors, remote sensors, GIS and interactive web platforms. In addition, different data collection strategies were examined, ranging from crowdsourcing campaigns to training local community workers.

6

A. How Can ICT Be Culturally Appropriate? Participants stressed that ICTD interventions need to consider local context carefully—in particular, to take into account literacy levels and language barriers. For example, one participant asked what ‘tech literacy’ really meant in a place where there were 3 languages; one based on the Roman alphabet that reads left-to-right; another in Arabic that reads right-to-left and a third that is glyph-based. Some programs chose to simplify ‘tech literacy’ creatively using the already well-understood meaning in one, two, three and four bars of cell phone reception as a way of communicating the quantity of the harvest on a given day. While writing out an exact measurement proved difficult, the harvest quantity was instead translated into a rough estimate based on the 1-4 bar symbol and this was texted instead. The need to adapt to local context also bumps up against practical cost considerations. Multimedia was tipped as an interesting tool for overcoming some of the literacy barriers in ICT: one participant stressed that the use of voice-based systems might work better in some contexts where there is a high illiteracy rate. However, currently voice-based systems are prohibitively expensive in most contexts, although costs will likely decrease with time making this a potential future strategy. It is important to work with the local community to develop a method of communication (that might involve some pattern recognition) that works for them—and these solutions will not necessarily come from the outside. The invention of a technology is just one step in a long process of adapting it to the local needs and context. Moreover, it is impossible to train someone once and assume they will adapt to a new method without continued reinforcement and support. It cannot be assumed that you can “add technology and stir.” Intermediaries, such as NGOs or agricultural extension workers, can play a critical role in translating technology to the local level and supporting sustained and widespread use of new technologies. Community involvement in the design stage was illustrated by the work of an NGO called Dimagi, which tries to develop locally appropriate solutions: what they call “designing under the mango tree.” Although Dimagi is a software company, more than half of the staff are not software developers, but rather a mix of development workers, sociologists and people from other disciplines. In addition, a good proportion of the staff is also based in different parts of the world, enabling local adaptation of their software platforms. They developed a Free and Open Source mobile platform for health care organizations called CommCare. Dimagi is often hired to adapt the technology to a specific organization (but a group can modify it themselves if they have the capacity as it is an open software system); it can be adapted for different purposes and languages. The importance of Free and Open Software is highlighted when local adaptation is critical. The technology, CommCare, enables healthcare workers to do their jobs more efficiently and effectively and enables the organizations to monitor their work better as well. Notably, the goal and end-users in this example are different from some of the other examples we discussed. Here, the organization is adopting the technology to improve its own efficiency and work-flow and is having its own trained workers use it. They are not attempting to crowd-source information from the public—which sidesteps some of the challenges around illiteracy, cultural appropriateness and incentives to participate. There are practical ‘realities of the ground’ that need to be considered, both as enablers and constraints to the use of ICTs for meeting FEW needs. There was wide recognition that what is implementable in urban settings can often be more problematic in rural areas. For example, many areas in Africa lack access to the electricity necessary to charge mobile phones. In fact, it is the ‘mobile revolution’ that has galvanized solar technologies on the continent—with the development of solar-

7

powered mobile phone chargers (e.g., Readyset Solar Charger1) now being used to charge other devices. Thus, the presence of one technology that depended on another provided the perfect incentive for innovation that also met energy needs. Hence constraints have the potential to lead to opportunities in the ICT sector, by providing incentives for innovation. ICT invention is still limited by local capacity—both human and infrastructural—in the countries that could make the most use of these technologies. A comment from a participant referred to the need for ‘indigenous innovation’ together with the local adaptation of ICT tools to meet local needs. Addressing this call for developing ‘indigenous’ ICTs requires a separate discussion on capacity-building to meet this challenge. Nevertheless, it formed an important thread through many of the other discussion that arose regarding adapting ICT to local contexts. Moreover, when indigenous innovation evolves organically, it should be supported and not blocked by external actors.

B. What Are the Incentives to Engage? Different strategies exist for gathering information—using crowdsourcing, using trained workers, using automated or remote technologies. The appropriate strategy depends largely on the goal of the project and the local demand it is filling. But other factors also play a critical role in determining the appropriate strategy. Some of these factors identified in the conference include: trust in government and other officials; local cultural rules and norms; illiteracy; and ‘demystifying’ data among others. The workshop participants discussed and debated the differing incentives that encouraged people to report data. It is sometimes assumed that crowdsourcing will work—yet it does not necessarily yield results as people do not input ‘data’ into the system for a set of reasons from time constraints to mistrust of how the data will be used. Whether crowd-sourcing strategies work depends a lot on the incentive structure to provide the data, as well as opportunity costs (often time), trust and other issues. When thinking about incentives and how to structure a program, it is critical to consider a demand-driven strategy that considers the need of the data users and the needs of the people providing the data. Incentives to participate can also be in the form of material benefits, such as cell-phone minutes recharged to an account based on participation in the project. In the public health world, there is a growing use of the ‘crowdseeding’ concept, which means having certain reliable people from whom you can consistently collect information. Effective crowdseeding requires that there be fair representation of a given population. In addition, ‘sentinel’ sites can be established from which data is collected on a regular basis. Despite the recognized limitations of data collection through crowdsourcing, there was also optimism for its role and potential. Several examples from Nigeria were given. In one case, someone concerned with how bad the traffic was in Lagos started re-tweeting traffic tweets (www.twitter.com/Gidi_Traffic). This turned out to be of great interest to locals, who began to change their driving patterns in order to avoid the traffic. Soon, lots of people were sending in texts and Nokia now partners with it so that an app is now customized for its Asa brand of phones where the tweets are available via push notifications. In another case, an application enabled people to report when electricity is on in a given location, and that information can be shared with others. The application would then take this information to predict when electricity could be expected in certain locations.

1 http://singularityhub.com/2012/08/20/readyset-solar-charger-successful-in-africa-now-headed-to-us/

8

In some instances, the provider of a technology has goals that are different from the community that is the target of the technology (which could be described as “split agency”). For example, a donor may want to provide a power source so that children can charge their school laptops. However, the community may want to use that power source to charge refrigerators, TVs or other technology. In some cases, donors have sought to change the power source to DC-only so that it can only charge the laptops. However, there are numerous other strategies that could exist for changing and regulating behavior.2 Overcoming some of the potential negative returns on reporting data are difficult to address. For example, data regarding diseased crops is very important, but farmers are less likely to report that their crop is diseased if they think they will be ordered to destroy it as a management strategy for containing the outbreak. In this case, incentives that outweigh the potentially negative repercussions need to be put in place. However, where consumers see direct positive results from their reporting, e.g., through sharing the price of drugs, the overall cost came down as consumers became aware that they were paying too much in some instances. This provided the incentive to report. Another drug-related example included a mechanism for verifying drugs by texting a code on the product through to a central database. These types of incentives can overcome cultural biases against reporting because of the direct benefits consumers receive, thereby institutionalizing the concept of reporting. The link between reporting and how the resulting information is used needs to be made explicit. For example, in China, the installation of automated sensors in wastewater outflows enabled local environmental officials to better monitor and collect pollution fees. Better communication of the benefits of reporting should be accompanied by managing expectations. Sometimes, the benefits of reporting data are indirect. For instance, if public services in a city are bad, one potential solution may be to develop a crowdsourcing platform so that citizens can report on the poor quality of services; however, if the services are not ultimately fixed, then the public may not have an incentive to continue to provide feedback. The best strategy depends on the rationale behind collecting the data. Is it for advocacy purposes or to raise awareness about a problem? Or is it so that the organizations providing the services can make their operations more efficient? Making explicit the different results that can be expected from providing data is vital for managing expectations and maintaining community buy-in to a project. Managing expectations also reinforces trust in the system, which is critical for long-term success. Finally, the power of ICT data is often in the updating, not merely in baseline recording. Real-time data by definition happens in real-time and so for it to be useful, it requires constant updating. The longevity of a reporting process is therefore critical and this is why creating the necessary buy-in of those reporting through incentives is critical.

C. Is Technology Socially Neutral or Can It Exacerbate Inequalities? The common perception is that technology can be democratizing, but that is not necessarily the case. In practice, technology creates “spikes” where access to the technology is concentrated. Technology is not inherently participatory or equitable. It can sharpen and exacerbate existing divides in a community. For example, those with greater resources may be able to co-opt the technology and receive a greater share of the benefits thus exacerbating local inequalities. Those providing information do not necessarily benefit from the information collected. An example was given of a community in the Amazon that agreed to participate in a project where they would send information about malaria prevalence; yet, the project was constructed without the real needs of the local people in mind. They were supposed to provide “sweat equity” to install the satellite tower and then send 2 See, for example, institutional arrangements analyzed through Regulation Theory (Spiller and Vogelsang 1997).

9

in reported cases of malaria; yet, the organization coordinating the project was not set up to send medicine or other help once malaria was reported. It is critical to recognize that when working with many marginalized communities, ICT can reinforce existing power dynamics. In such situations, it is important to identify the power brokers of the information flow and what the outcomes of the data collection will be, i.e., what is the response and who is expected to respond once the data have been collected? This emphasizes the concept of an ‘end-to-end’ project design and talks specifically to projects in rural areas where ICT allows for the flow of information more easily than ever before, but does not necessarily improve the ability to respond to this data. “Social networks” were not created by the internet. Traditional, relationship-based social networks have always existed, at least at the local level. Technology is enhancing traditional forms of communication, enabling people to create much broader networks with a much more diverse and geographically broad group. However, technology can also displace traditional social networks, and these impacts are not inherently positive. For example, if a woman in a village does not have her own cellphone and/or is not literate, will she benefit from texts being sent about water, health, weather, etc., or will she continue to rely on her neighbors and friends for information? This issue of the “substitutability” of technology for existing mechanisms or an “anthropology of technology” is important to bear in mind when critically assessing ICT interventions in communities.3 It was also argued that ICTD community suffers from too many pilot projects (“pilotitis”)—and the transaction costs are high because there are real (sometimes negative) impacts on the local communities. Yet, one participant suggested that the mindset of technology developers and engineers may need to be changed: they should first go to the field and understand the problem and the demand before attempting to create a solution. ICT is not a ‘once-size-fits-all’ solution.

D. To What Degree Does “Small Infrastructure” (ICT) Depend on “Big Infrastructure”? Professor Juma argued that one of the problems hindering economic growth in the developing world is the absence of adequate infrastructure. He noted that currently, infrastructure projects are designed mainly to provide specific services—transport, water, sanitation, energy, telecommunications and research facilities—but that infrastructure development can also be designed as a foundation for technological innovation thereby setting the stage for technological ‘leap-frogging.’ ICT provides just such an opportunity for bypassing certain technological stages of communication technology (e.g., landline telephones) as well as increasing people’s capacity. For example, it can enhance entrepreneurial capacity and skills when farmers are provided with the means not only to produce food, but to interact directly with markets through a mobile device. ICT can thus act as a catalyst for development. However, there is a catch 22 situation in that ‘small infrastructure’ (ICT) depend on the services provided by larger infrastructure (network cables/satellites). One key question that emerged from the workshop is to what degree does small infrastructure (i.e., ICT) still depend on big infrastructure (i.e., roads, telecom cables, satellite services, etc.)? ICT is associated with empowerment through the sharing of information, which can be a powerful tool for development. However, actual development relies not only on the presence of information, but a resulting action that often requires ‘big infrastructure’ for service provision. For example, in contrast to the example of mobile phones driving the development of solar powered charging stations, the relationship between an increase in mobile phones in slums and an associated increased ability for people to communicate and provide information about the state of their living conditions does not automatically

3 For example, see Pfaffenberger (1988).

10

translate into an increase in ‘big infrastructure’ provision (e.g., sanitation services). An increase in one technology over time does not equate to the development of other technologies, especially if ‘development’ is predicated on ‘large infrastructure.’ The relationship between technological innovation and development is difficult to measure, yet important to understand. What guidelines can be established that can assess the relationship between the two and how inter-dependent they are? For example, participants questioned whether it makes sense to invest resources in ICTD when there is not the physical infrastructure to make it really functional. One participant gave the example of the rise of mHealth applications in East Africa. On the one hand, it is great to have an electronic pen and a device that reminds you when a certain procedure needs to be done. However, if a villager cannot get to the hospital due to poor roads or lack of transportation, then it is largely useless information unless it can be used to build an argument for building larger infrastructure like roads and hospitals that deliver services directly. In many ways, we can think about 4 “layers” of technology with which ICTD interacts. The first is the need for big infrastructure like submarine cables. For example, the number of submarine cables connecting Africa to other parts of the world rose dramatically in recent years—and was attributed in part to the World Cup being held in South Africa, which drove the need for good broadband service for high quality broadcasting of the matches to Europe. Second is technology regulation and policy, which, for example, governs where electricity supplies will go and how it will be regulated. Third are remote and automated technologies, which can provide independent and objective gathering of information. Fourth is the role of ‘human sensors’—i.e., relying on people to gather information. These four layers map onto our understanding of ICT having four different pathways (See Figures 1 and 2).

Figure 1: This is an illustration of the 4 pathways of ICT (Source: Revi Sterling).

11

Figure 2: Examples of ICTD occurring spanning the local to the global levels (Source: Revi Sterling).

E. How Can Culturally Appropriate Technology Be Developed? The workshop also discussed new ways to generate technology appropriate for development. For example, hackathons provide a venue to connect the tech and design community with development practitioners. It can be described as “speed dating”—you see if there is a fit between a need and someone with the skills to fill it. Similar opportunities for greater interaction and knowledge exchange between these two communities is important if useful ICT tools are to become more widespread. A growing number of international institutions are participating and occasionally organizing software competition like hackathons. For example, the World Bank recently organized a series of hackathons to generate new ideas for solving traditional water and sanitation problems. The World Bank has traditionally focused on big infrastructure and governance issues, working closely with governments. The hackathon was an opportunity to bridge sectors and invite developers and civil society to brainstorm together. One of the key things about the Water hackathon was that it was practitioner-focused: the World Bank spent time soliciting information from governments about problems they needed to resolve. It also became an entry-point for opening up of data; one agency released a host of data about water that had not been public before. This is yet another example of ICT serving a boundary spanning role between the World Bank and local governments. The shared understanding the project created also opened the space for further trust and data sharing. Hackathons can generate ideas that transform the nature of traditional procurement channels. For example, a Kenyan water utility wanted to enhance citizen feedback about its services. When the utility put out a bid for service providers, an application that had been developed by a group of Kenyan developers participating in the World Bank hackathon won. The developers were hired as individual consultants to the utility (since they were not yet formed as a company) and have been working to adapt it to the needs of the utility. A pilot project in Nairobi’s Western District is planned for this summer (2013). Hackathons can also enable problem identification statements to be archived and worked on at a different time. For example, at a hackathon organized by Random Hacks of Kindness in Montreal, a group of developers became interested in a problem posed at a prior hackathon—and that was still listed in the repository of problems. As a result, the group mWater was created, which is using mobile devices

12

to test and track water quality. These prototypes, however, can only materialize further if there is adequate funding. Hackathons can capitalize on local talent and community buy-in. For example, one participant described a Digital Democracy hack-a-thon where they went to Port au Prince to develop technologies related to gender-based violence; the US team of software designers served in an advisory role but the local groups drove the mission and design ideas. Everything was also done in the local language. One participant critiqued hackathons for not having a quality bar. At a hackathon, there can be customization of existing tools, but too often, there is only re-invention. For example, Free and Open Source Software licensing enables other groups to adapt existing technology but the tendency is to build from scratch. That said, others participants suggested that the point is to innovate and encourage communication—and that there are some natural filters to ensure quality. For example, the reality is that the vast majority of applications developed at a hackathon do not actually materialize. They require significant funding to make a demo and roll it out. The challenges of adaptation at different scales were discussed. When working with scientists thinking about collecting ‘big data,’ there can be a hesitancy to try something new because it is not yet perfect. In this context, there was a call to get the technology out there—and be prepared to adapt. Interestingly, one of the participants in the technology world suggested that there is a common perception among technologists that technology is a way of development—so why and how should it interact with traditional development sectors?

IV. What Are the Ethical Implications of the Rise of ICTD? The workshop participants grappled with ethical issues associated with data collection and preservation strategies. ICT has changed the nature of research and information flows. Universities and big organizations working in the humanitarian or human rights context, for example, the Red Cross, Human Rights Watch or Doctors without Borders, have internal procedures that scrutinize the potential ethical implications of their work—such as through university Independent Review Boards. There is not yet a similar type of verification in the emerging ICT for development/humanitarian work context. For example, consider that an NGO decides to map and report all of the rapes reported in a village. The NGO has information about the location and the event—but also the name of the alleged perpetrator. What information should the NGO post online? There are a host of reasons why the alleged perpetrators name should not be posted—but if it does not post the information it has, is that censorship? How can organizations (universities or big NGOs) share their internal IRB-like standards and procedures and will ICTD groups realize the need to go through this thinking? The ethical obligations may depend on who is doing the collecting. For example, the ethical responsibilities of a researcher (who may be required to withhold identity) may be different from those of a journalist (who may want to report the identity of someone). Consider an application that might warn people of an impending disaster, such as a flood or nearby military action. First, there is a question about whether this information is reliable. Who is verifying how this information is gathered? Is there a way to cross-check this info? Second, such information could cause mass displacement—and is there an obligation for the provider of information to consider these ripple effects? This information may simply be enhancing an incumbent form of information communication, such as a radio alert, or it could be completely novel.

13

The fact that ICT makes it possible to communicate information horizontally between people as opposed to simply vertically (from government to the people) opens up the door to a host of potential questions about data reliability and ethics. Sharing of information horizontally can bypass the traditional institutions designed to deal with the vetting of information. One stark example given was of a map of all rapes committed in the Congo. The information is online—but in order for action to be taken, it would still need to go through the local channels by being reported to the police, and then perhaps going to the courts in a traditional ‘vertical’ system of reporting. The theory behind making this information accessible is to put pressure on policymakers—yet things do not appear to change (at least not fast). Much of the information now available online may have once been available in a big report produced by the United Nations (UN) or other agencies. Does the fact that this information is now easily available online change the ethical obligations of those who have easier access to it? There are so many applications being developed—and there is no way of verifying what data is accurate and what is not. Moreover, with lots of applications being developed for the same purpose, who is focusing on ensuring that these applications actually work together—and can they be integrated in one platform? This also led to a discussion about privacy, confidentiality and security as more people share more data.4 What is going to be done with the data that is now gathered and how long will it be kept? Will this information lead to ‘big brother’ surveillance? For example, big data has unintended uses. For example, a bank might be able to see if your Facebook friends repay their loans before giving you a loan. Part of the analysis is based on the relationship a society has with its government and the trust that is there. But given that there are a lot of other private and non-governmental actors who can use this technology, the discussion must go beyond just governments.

V. ICT Across Scales: How Does ICTD Inform Policy-Making? The participants also discussed the different scales of information and the way in which big data (usually gathered by governments, transnational organizations or private companies) were linked up to small data (i.e., small ICTD projects often run by NGOs).

A. The Advent of “Big Data” We increasingly live in an age of big data, which raises a host of challenging issues. The workshop participants used the term “big data” as a short-hand to refer to the large amount of information that is collected on a regular basis by governments, multi-lateral institutions, private companies, researchers, etc.5 These data come from everywhere—sensors used to gather climate information, posts to social media sites, digital pictures and videos online, transaction records of online purchases and cell phone GPS signals to name a few. Big data requires new legal and other institutional capacity to address issues of confidentiality, identity, privacy and copyright. There is so much data being generated and collected that we do not yet understand how to use it effectively. Furthermore, big data requires a new research design for data analysis.

4 For further information, see Schmidt and Cohen (2013). 5 For example, IBM defines big data as follows: “Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone. This data comes from everywhere: sensors used to gather climate information, posts to social media sites, digital pictures and videos, purchase transaction records, and cell phone GPS signals to name a few. This data is big data.” See http://www-01.ibm.com/software/in/data/bigdata/.

14

The workshop underscored that better data can improve decision-making tremendously, but only where decision-makers favor policy that is evidence-based and context specific. From a technological point of view, before the wide innovation and diffusion of ICT, our society’s capacity to monitor, collect and process data was quite limited. Data were usually gathered, analyzed and disclosed by professional government institutions equipped with expensive technologies and sufficient resources. ICT is making it easier for these traditional agencies to collect data on a more regular basis. However, the technical capacity to do so is only part of the equation. The potential power of ICT lies also on the political willingness to use that data for better evidence-based, policy-making. Three categories of ‘big data’ that were seen as most relevant to policy-makers seeking to understand the potential of ICT for development are:

1. Traditional statistics from international organizations such as the FAOSTAT.6 Here, the data gathering capacity of large, international organizations depends largely on the provision of data by their member states. Although the accuracy of the data is often questionable as it relies on country reporting, it is largely the main source of data for large-scale analysis of socio-economic trends.

2. Open government data. The ICT revolution has resulted in it being possible to ‘open up’ government data. Open Government Data usually refers to public records (e.g., on transport, infrastructure, education and environment) that can be freely used and redistributed by anyone—either for free or at marginal cost. While such transparency is generally encouraged, it also raises issues of privacy and security.

3. Real-time data. The major revolution that ICT brings to the field of data collection is its ability to report data in real-time; rather than having to wait for all the data to be gathered at discrete points, real-time data allows for monitoring situations as they actually occur as well as the analysis of trends occurring in real-time, impacting time series analysis fundamentally.

A section of the United Nations, UN Global Pulse, is particularly interested in the third type of data and is partnering with the private sector (often the collector of these data).7 UN Global Pulse is exploring new digitized ways of mapping vulnerabilities. The idea behind this initiative is simple—once we know what signals to listen for, we will be able to ‘take the pulse’ of vulnerable communities. The importance particularly lies on the engagement of citizens in the development process. The evolution from the one-way, static era of Web 1.0 to Web 2.0 has enabled us to have new sources of information for big data through two-way communication, such as Twitter and Facebook. For example, the Google Flu Trends8 reflects a change in how we approach data where Google uses aggregated search data to estimate flu activity. Another example is using newspaper and other media reports on food to track food vulnerability or the UN Global Pulse project that scrapes online prices of bread in 6 Latin American countries to create the e-bread index.9 The Web is evolving from a “Web of linked documents” into a “Web of linked data.” Linked data is about connecting pieces of related data and information coming from different sources (e.g., information systems and databases). Linked open data initiatives are expected to promote transparency, foster collaboration across government and beyond, allow the creation of new, innovative, added-value services, and improve the quality of decision-making. The potential of Web 3.0 opens up new possibilities by

6 http://faostat.fao.org/ 7 http://www.unglobalpulse.org/ 8 http://www.google.org/flutrends/us/#US 9 http://www.unglobalpulse.org/projects/comparing-global-prices-local-products-real-time-e-pricing-bread

15

allowing data to link with data (linked data) such as through the case study10 by the Interoperability Solutions for European Public Administrations.11 Big data has the potential to strengthen links between science and policy-making. For example, from a climate science perspective, there is a gap in the availability of socio-economic data at a relevant level to be included into realistic scenario formulations for better projections of climate impacts. Big data has the potential to help fill this gap with real-time information at the requisite level and in the necessary quantity to make projections that much more accurate. Furthermore, in terms of climate science, lots of excellent information exists yet it requires sophisticated software to understand and process it—which makes it inaccessible to policymakers and the public. ICT also has a transformative potential with respect to “big data.” Historically, the general public was a passive receiver of official data (despite often being the source of it). The ICT revolution theoretically enables ordinary people to create and share data by themselves, thus endowing them with new political powers. While more and more technological possibilities of acquiring data are being created with affordable costs, data may no longer be a privilege by special agencies but commonwealth of the public. Ultimately, the involvement of local communities in data collection and monitoring has great potential to increase the transparency of processes. However, as a practical matter, this potential may not always be translated into reality.

B. Are All Data Equal? The Challenges of Accuracy, Verification and Analysis Data are not created equal; they require context. Data can come in a variety of forms, e.g., a series of numbers, sentences, pictures, videos and animations. The diversity of data means that there are increased ways of communicating data, enhancing their ability to meet the needs of different people: the young and the old, the well-educated and the illiterate. However, we need to accept that these data are “messy”—for example, Google search histories can provide a lot of information, even with misspellings, but it is difficult to identify the reason why such trends occur—hence, the need for greater context to enable us to explain why certain trends are being seen. Context gives us the ability to try to attribute causation rather than identifying mere correlation. The source of the data is also important; data available publically (Twitter) are often far less reliable than the data from the private sector (Google searches), but using private data once again brings up issues of privacy and confidentiality. Some private data is hard to access, so that the distribution of the data can be both spatially and temporally unequal. Is it possible to determine the national mood by viewing Twitter feeds? As one participant said, “the word ‘accurate’ should not be followed by the word ‘data’.” However, data may not be accurate, but they can still be useful if processed correctly. The end users are often concerned about the quality and reliability of big data and data gathered in ICTD projects. There is always some level of uncertainty and the question is to determine the minimum level of uncertainty that is acceptable. Furthermore, it is important when dealing with “big data” that one is making up for quality with quantity: the data collected are not a statistical ‘sample’ of a group, but rather just a lot of data points, which requires nuanced statistical approaches in order to make them ‘useful.’ Empirical researchers are often concerned not only with knowing how data were collected (as well as the context) in order to draw conclusions from data analysis, but are also concerned about being able to replicate the data collection for verification, tracking trends, future research, etc. (King 1995). Making use of “big data” can complicate the scientific method and requires the development of new norms for conducting and replicating research that maintains scientific rigor.

10 https://joinup.ec.europa.eu/sites/default/files/D4.3.2_Case_Study_Linked_Data_eGov.pdf 11 http://ec.europa.eu/isa/

16

There is often a mismatch between the question decision-makers want answered and the data that is readily available. ICT can to a certain extent remedy this problem, but it brings with it its own ailments in terms of sample and the form that the data take. These often need to be processed with sophisticated software in order to be understood—usually with some form of visualization in the form of graphs. The mere existence of the data is insufficient; it requires software and human capacity to make sense or ‘translate’ it into something useful. Furthermore, the process of ‘cleaning’ and even verifying data often results in removing anomalies or chopping off long tails in distributions. However it is often in the anomalies where the most interesting things are happening. Identifying what is useful and what is ‘noise’ requires sophisticated analysis. Verifying data accuracy is a major challenge.12 This is a common problem in the disaster response context where things are happening so fast on the ground. One way is to try to verify it independently. But it may be that some data needs to be gathered from an official source to be accurate. Remote and automated sensing technologies provide a way to verify, thereby reducing human error and corruption. For example, in China, automated monitoring of pollution from industrial and wastewater plants is helping to improve environmental regulation and reduce opportunities for corruption, especially when regulatory agencies do not have staff or capacity to effectively monitor. Making data immediately available to the public can also create an added challenge if the data set is refined or cleaned up during the processing stage. Thus, it can be possible to have multiple versions of the same data set in circulation, which can create quality problems. This ‘versioning’ of data can have implications for transparency and decision-making. ICT offers the potential to verify information that has been collected manually. For example, an example was given of geo-tagging the locations visited by volunteer data collectors to verify that they had been physically present at the field site. This prompted a comment about whether this was akin to the organization spying on volunteers; but as these were willing volunteers doing a task, this was seen in that context as more appropriate (see the ethical discussion below for more on this point). Unreliable data can lead to a “decision gap.” For example, with the increasing number of crisis maps being produced, what should a policymaker do with this information if there are conflicting reports? More data do not automatically translate into more trust in the data due to the aforementioned need to analyze it before it is useful for decision-makers. This challenge is analogous to another challenge: that of filtering the data we are now overloaded with. What processes need to be in place to filter information appropriately? Sometimes the meaningful analysis from big data can only provide the range of acceptable values or distributions, such as threshold of minimum acceptable range, the probability distribution functions (PDFs) or windows of uncertainty. The interpretation of the analysis for decision-makers is yet another stage in the process and following this is how to prioritize interventions once the data have been analyzed.

C. Does More Data Lead to Better Decision-making? The workshop participants discussed whether the accumulation of more data through ICT leads to better decision-making. Several stressed that there appears to be a general lack of evidence-based decision making in the policy world. The reasons for this trend were debated, with some emphasizing the challenges of translating complicated scientific information into useful policy prescriptions. One participant provocatively suggested that the reticence to embrace evidence-based decision-making was due in part to the fact that most politicians are not trained as scientists. Yet others (including those who had served in government) suggested that political office changes a person’s priorities—including those who are scientists. Politicians see the evidence that they want to see and will not rely on good data if it

12 For more on methods for data verification see King (1997).

17

does not show the desired results. Thus, without greater political openness to viewing the evidence objectively, we cannot assume that simply adding more ‘evidence’ through ICT will improve outcomes. An example from the Nile Basin Initiative highlighted the important role that technology can play in building trust and fostering collective action. The Nile Basin Initiative (NBI) is an inter-governmental group that is trying to develop an appropriate management plan for the Nile. The river has a unique and diverse system and huge potential for improved use and development. However, the challenge is in trying to get the countries to see the benefit of cooperating across boundaries. For example, simply because of evapotranspiration rates, it would be possible to save a large volume of water if a dam was built in Ethiopia instead of Egypt. Historically, each country would each bring an expert from its own country to provide evidence because there was no trust. Now, the NBI has provided opportunities for scientists to use automated sensors to provide real-time flow data—and this is building trust in the process. By simply talking about the data and making it available, scientists are slowly gaining the interest and trust of policymakers—which might lead to more evidence-based decision-making in the future. The ability of ICT to make data available can lead to increased transparency in the data collection and analysis process thereby building trust. The critical role of trust in sharing data and information was further highlighted in other contexts. In other words, the more that there is trust in the system, the more that information is likely to be shared. But at the same time, ICT can be used as a forum for building trust as different participants come together around ICT to build common understandings of problems and possible solutions. In this respect, ICT can be seen as “boundary spanning,” identified in the sustainability literature as critical for designing and implementing effective programs linking knowledge with action (Cash et. al. 2003). As such, if implemented equally, it can also help decision-makers with prioritization of projects by providing information on what the concerns of citizens really are and where these issues are occurring. This ‘democratization of data’ through increased citizen participation can be seen as an incentive for politicians to engage with real evidence-based decision-making if their constituents are engaged in data reporting.

D. Can ICT Enable the Forecasting of Future Events? Questions were raised about the ability to take big data and disaggregate it so that it can lead to more effective and targeted decision-making. Many of the participants stressed that the rapid rise in data collection and mapping through ICT has not necessarily improved our ability to predict new events and be prescriptive about them. A key challenge is that it takes a long time to collect, process and analyze data properly. For example, one researcher who was involved in a project in Iraq said that in the wake of the fall of Saddam Hussein, they collected a lot of data; but it was only a year and a half later that they could concretely say that de-Baathification was a bad idea—and by that time, it was too late to do anything. Food vulnerability is very difficult to map and predict because it is not simply about food production. It is also about access to markets, understanding dietary habits, income of the household, nutrition and cultural traditions (like who eats first, especially in times of scarcity where there may not be much left-over for those who eat last). Food vulnerability is often of greatest concern in fragile countries—and so there are now some nascent efforts to try to integrate data about food with data about peacekeeping efforts. To be able to effectively predict vulnerability to disasters, policymakers need to better understand decision-making at the individual level. Sometimes data can tell us very surprising things. For example, research in the Philippines focused on people who are displaced by disasters revealed that those

18

who return actually fare worse than those who move to other cities (where they may have support networks) or who are in camps (where they continue to receive government or foreign assistance). Making accurate predictions about vulnerability is difficult due to the challenges of accurately analyzing existing data trends and incorporating non-linearity and uncertainty into prediction models. In order to forecast one needs to be able to read from trends in the data that a certain event is likely to occur. But what sort of trend shows us this? Is it sudden and abrupt—or is it a collection of smaller spikes that culminate in a big spike? The point is that often we do not know enough about what the trends in data show us (leaving aside the quality of the data itself) in order to be able to forecast in social environments in the way that we can do in the natural environment. In general, we have not yet studied the social system and its feedbacks as well as we have done in the physical environment, largely due to a lack of large-scale data. For example, in retrospect, a hike in food prices may have been an accurate indicator of an impending food crisis; but given the numerous other variables at play, it may not be a consistent forecaster for all situations. More real-time data needs to be analyzed to recognize trends that could lead towards a ‘tipping point’ or crisis situation and to make this prediction before the event has already occurred. The first step on this road is being able to back-cast using the data that is now available to us. Translating these relationships into forecasts will require much more analysis and a better appreciation of uncertainty before they can be useful.

VI. Framing the Big Picture: What Is the Role of Theory?

A. What Role Do Institutions Play in Promoting ICTD? Institutions play a key role in the creation, use and effectiveness of ICTD. Institutions are the rules of the game in society or, more formally, the humanly devised constraints that shape human action (North 1990). The constraints can be formal (such as laws and regulations) or informal (such as conventions and codes of behavior), but key to the definition is that there is a societal sanction for transgression of both rules and norms. They can also be thought of as structures of meaning including symbols that explain and justify behavior (March and Olsen 2004). Institutions are important to reduce transaction costs and to provide solutions to collective action problems (North 1987). Therefore institutions evolve to fulfill these needs and mitigate market failures. Institutions function at multiple levels in ICTD. First, institutions define the types of questions and problems that are addressed by ICT in a development context. Second, they determine the level of trust and transparency between relevant actors in an ICTD project. Third, institutions underlie any regulatory or legal framework governing the use of ICTD, which can often differ across countries. Fourth, institutions also shape local cultural and social norms that will have strong interactional affects with how ICTD is perceived and used. One critical role for institutions in ICTD is their role in regulating “small data” gathered through ICTD and “big data.” Regulations at the institutional level will have advantages and disadvantages. The pros are (1) to reduce the risk of manipulation of data, (2) to decrease the gap between the rich and the poor (i.e., change in power structure), (3) to protect privacy and consent, and (4) to minimize the chance of censorship and increase transparency. The cons are (1) to prevent innovation by imposing too many constraints on innovators and (2) the opportunity costs of pursuing ICT solutions that are ineffective or less suitable than more mundane solutions. Table 1 represents an attempt to develop a framework for considering the “risk” posed by greater institutionalization and regulation of ICT. In developing the table, the group considered its application to the food sector (e.g., mobile crop forecast), energy sector (e.g., smart grid) and water sector (e.g., the

19

Nile Basin Initiative). This suggested framework does not intend to provide answers to individual cases, but rather to provide an example of an attempt to assess the degree of impacts of institutional regulations to ICTs at different sectors. Each pro and con is assessed on a three-point indicative scale: “H” for High, “M” for Medium, and “L” for Low. Values on this scale were assigned by one of the break-out groups. Table 1: Degree of significance on FEW sectors by implementing institutional regulations

Pros of Regulation/Institutionalization Cons of Regulation/Institutionalization

P1 P2 P3 P4 C1 C2 Food M H M L M H

Energy L L H L H L Water H H M L M M

P1: to reduce the risk of manipulation of data P2: to decrease the gap between the rich and the poor P3: to protect privacy and consent P4: to minimize the chance of high censorship C1: to prevent innovation by imposing too much constraint C2: to cause ineffectiveness

B. What Is the Role of Theory and How Does It Inform Practice? The ability of ICT to provide data in a variety of forms and through diverse processes has resulted in a concomitant push to make use of this data, for everything from advocacy to decision-making, large-scale modeling to local service delivery. However, what became evident over the course of the workshop was that there was no theory accompanying this use of technology. Very few norms accompanied the devolution of ICTs for data collection apart from those that already existed for the collection of information. Issues to do with ethics have been raised, but there is no widespread model of good practice; rather there has been a ‘hype cycle’ whereby the ease of deployment and the potential of ICT has become self-reinforcing without significant recourse being given to the institutions that seek to regulate the practice. Significantly, in a world where there is an increased pressure from donors and funding organization to provide evidence of project successes, the ability of ICT to generate data and provide this type of evidence was identified as a potential reason as to why the use of ICT spread so fast and so widely. Following the MDGs, in the FEW sectors, projects became ‘outcomes’ oriented with a particular emphasis on having to measure these ‘outcomes’ (i.e., x number of water taps built; y number of children fed in a school program). The potential for ICT to provide the data of these ‘outcomes’ made them almost a panacea of evaluation tools. Added to this, the research community in the age of globalization began to recognize the complexity of FEW needs interacting with social systems. This required a better resolution of data in order to analyses processes of change that cut across multiple levels between the local and the global. However, as has been made explicit in this document, there can be benefits and drawbacks from using data that are currently being collected by various organizations using ICTs. Some of the benefits include its potential to be real-time, the speed at which it can be gathered, its potential for amplification and scaling-up, its ability to increase transparency and trust by being open and potentially participatory and its capacity for experimentation with large datasets. Some of the drawbacks include its questionable accuracy, the lack of context for some of the data generated, an inability to control the ‘sample’ in crowd-sourcing, an inability to appreciate trends and issues of accountability, security and privacy.

20

Given these trade-offs, several workshop participants proposed a schematic (Figure 3) to conceptualize how theory could talk to practice in the context of evidence-based decision making. The group sought to emphasize that instead of a linear process whereby data gets collected, analyzed and then informs decision-making, ICT-based data is produced in unique contexts and therefore requires novel theory to inform its analysis: with the caveat that the theory needs to be informed by practice. If we assume ICT will increase access to vastly more data, then consideration of local context is all the more vital to informing sound data analysis and recommendations. By theory we refer to a means by which the ‘black box’ of data can be made more transparent to those providing it as well as those using it—a means of translating the data. It also refers to the need for ethical considerations to be made more explicit in this process. The role of academics was discussed at length because in the ICTD field, it is often NGOs, governments or UN-type bodies who are collecting and using the data. The role of academics is often as observers who sit outside the system, and it was noted that there was a need to bridge the gaps between the different groups operating in the space depicted in Figure 3. This emphasis on ‘boundary management’ between knowledge and action echoes the call by Cash et al. (2003) for three key functions to be considered in the knowledge system: communication, translation and mediation. Based on the discussions, the optimal enabler for this role in ICT for FEW was seen to be a set of theories that could start to unpack the complex realities that ICT data present, in particular how to address the data trade-offs during analysis.

Figure 3: Schematic illustrating the role for theory that enables data analysis to take into account the

unique context within which ICT data is gathered and thereby the actions that should be taken based on the analysis of this data.

21

C. Is There a Need for Decision-support Tools? There is a critical need for decision support tools to help provide guidelines for effective ICTD implementation. One idea that emerged was to frame a hypothesis about the relationship between trust and the design of effective ICT strategies. Table 2 attempts to illustrate this relationship. It is by no means a complete analysis, but it illustrates the type of thinking that was spurred at the workshop. Table 2. Proposed hypothesis regarding the relationship of trust in the government and the design of ICT strategies.

Trust in Gov/Institutions Promoting ICT Strong

Trust in Gov/Institutions Promoting ICT Weak

Direct communication between data providers and program implementers

Preferred for accuracy, knowledge about contributors leading to better estimates of bias in data

Likely to have significant non-compliance if citizens lack trust in implementing agencies, which will undermine ICT tools

Anonymous crowed sourcing of data

Crowed-sourcing data may be effective in certain circumstances and often provides a cheaper solution if there is incentive for participation, but direct communication ICT strategies will usually provide more accurate and certain data

Crowd-sourcing data so that data providers remain anonymous is likely to be more effective where trust is limited

Green: Best option Orange-Yellow: Intermediate option Red: Bad option

VII. Conclusion This workshop provided a valuable opportunity to bring together practitioners and scholars to consider with a critical eye the role of ICTs in helping decision-makers meet food, energy, water, health and other important development needs. The first major theme that was emerged was the need for developers of ICT projects to consider the local context. The cross-cutting dialogue highlighted the challenges of developing culturally appropriate technology and the need to identify incentives that drive individuals to willingly provide data. While technology holds tremendous power to ‘lift all boats on a rising tide,’ it can exacerbate existing inequalities in society unless implemented appropriately. The second major theme discussed at the workshop focused on the ethical implications of the increased use of ICT in the development context. In particular, the increased ease with which large volumes of data can now be collected raise tremendous privacy, confidentiality and security concerns. Organizations looking to use ICT in development contexts need to consider potential ethical obligations that arise from collecting and storing sensitive information. Traditional ethical obligations associated with research also need to be re-examined in light of the changing nature of information flows. Third, the participants grappled with the changing nature of “big data”, a term that refers to the large amount of information that is collected on a regular basis by governments, multi-lateral institutions,

22

private companies, researchers, etc. While more and more information is being collected, it is not always equally reliable. Moreover, while all participants stressed the importance of data-driven decision-making, it is not clear that policy makers rely on data to the extent that they could. Finally, the workshop participants sought to link the practice-oriented conversation around the role of ICT in development with the broader academic literature, including institutional theory. The need to better link practice and theory was highlighted, and several hypotheses were generated that are included in this report to spur research in this direction. By bringing together technologists, policymakers and scholars, the workshop sought to identify critical areas for future research. A series of questions for future research were articulated, including on the following themes: the need to identify factors that lead to successful, demand-driven and culturally appropriate uses of ICT; the type of decision-support tools that can aid policy-makers seeking to implement ICTD projects; the link between “small” and “big” infrastructure; the role that an independent verification system for ICTD projects could play; factors that might lead to data-driven decision-making; and the ethical obligations that accompany the collection of data in an ICTD project. Overall, the workshop was a tremendous success, achieving its goal of bringing together practitioners and scholars to consider the ever-increasing role that ICTs will play in meeting global sustainable development challenges.

23

VIII. Appendices

A. List of Participants and Group Photo (in alphabetical order)

1. Petrine Addae, Master in Public Administration in International Development candidate, Harvard Kennedy School.

2. Césaire Ahanhanzo, Masters in Public Administration-Mid-Career candidate and Mason Fellow, Harvard Kennedy School; and policy and leadership experience from Benin.

3. Willow Brugh, Director, Geeks Without Bounds; co-organizer and facilitator of events such as Random Hacks of Kindness, SpaceApps Challenge and Konbit Technologie.

4. Dhairya Dalal, Business systems analyst for Harvard Information Technology (HUIT), and Masters in Software Engineering candidate, Harvard Extension School.

5. Bob Day, Founder and Co-Director, Non-Zero Sum Development; Member of Word Bank Expert Team working on Tanzanian Science, Technology & Higher Education Reform Program; and Special Advisor to the Mozambican Minister of Science and Technology.

6. Yohannes Gebretsadik, PhD student at the University of Colorado, Boulder, and Visiting Student at the MIT Joint Program on the Science and Policy of Global Change.

7. Pragun Goyal, Master of Science candidate at the Responsive Environment research group at MIT Media Lab.

8. Patricia Guardabassi, Sustainability Science Fellow (2012–2013), Harvard Kennedy School. (Co-Lead Organizer)

9. Alicia Harley, PhD candidate and Sustainability Science Fellow, Harvard Kennedy School. (Co-Organizer)

10. Calestous Juma, Professor of the Practice of International Development at Harvard Kennedy School of Government.

11. Eunjee Lee, Sustainability Science Fellow (2012–2015), Harvard Kennedy School. (Co-Organizer)

12. Angela Livino, Sustainability Science Fellow (2012-2013), Harvard Kennedy School.

13. Julia Cadaval Martins, Visiting Researcher, Harvard Law School.

14. Scott Moore, Sustainability Science Fellow (2012–2014), Harvard Kennedy School. (Co-Organizer)

15. Sharmila Murthy, Sustainability Science Fellow and Carr Center for Human Rights Policy Fellow (2012–2013), Harvard Kennedy School. (Co-Lead Organizer)

16. Egghead O’Seun Odewale, Fellow, Berkman Center for Internet & Society, Harvard University.

24

17. Mukhtar Abdi Ogle, Masters in Public Administration-Mid-Career candidate and Mason Fellow, Harvard Kennedy School; and former Principal Examinations Secretary, Kenya National Examinations Council, Ministry of Education (Kenya) (2012).

18. Nnenna Okoye, Program Analyst, Dimagi.com.

19. Laura Pereira, Sustainability Science Fellow (2012–2013), Harvard Kennedy School. (Co-Lead Organizer)

20. Daniel Shemie, Co-founder and project manager at mWater and former organizer of water-related ‘hackathons’ at World Bank.

21. Revi Sterling, Director of Information and Communication Technologies for Development (ICTD) at the Alliance for Technology, Learning and Society (ATLAS) at the University of Colorado Boulder.

22. Deniz Susar, United Nations Department of Economic and Social Affairs, who is currently serving as the Coordinator of the United Nations Public Administration Network (UNPAN) Management Unit

23. James Wescoat, Aga Khan Professor, Aga Khan Program for Islamic Architecture and Department of Urban Studies and Planning, School of Architecture and Planning, MIT.

24. Patrick Vinck, Research Scientist at the Harvard School of Public Health and Associate Faculty with the Harvard Humanitarian Initiative (HHI). pvinck@hsph.harvard.edu

25. Chao Zhang, Sustainability Science Fellow (2012–2013), Harvard Kennedy School. (Co-Organizer)

25

B. ICT for FEW Workshop: Original Agenda Friday, May 17 6:30pm Dinner at Harvard Faculty Club Informal introduction of each participant (name/institution/interest) Participants to write down a few questions/themes to discuss Saturday

Supper Talk: ICTs 4 FEW 4 ALL? by Bob Day (10-12 mins) There are critical issues for an inter-disciplinary research agenda addressing the role that ICTs can play in promoting ubiquitous and secured access to food, energy and water (FEW). In identifying and starting to address these, this workshop advocates some refreshing approaches: avoid technology push; harness multi-disciplinary and multi-sectoral collaboration to tackle complex problems, using systemic thinking; combine the strengths of bottom-up and top-down approaches, amongst others. However, there are also major challenges that cannot and should not be avoided. For instance: a paucity of evidenced-based thinking and policy formulation based on evidence; ICT's poor track record for sustainable impact; a lack of awareness of the role that ICTs can play at the policy and strategy levels. These few remarks will provide a context to consider the question being posed for the workshop: What role can the data generated through ICTs play in aiding decision-making to meet current and future FEW needs in the wake of climate change?

Saturday, May 18 8:00 Breakfast 8:45am Welcome Remarks

Professor Calestous Juma 9:10am Goals of this Workshop

Laura Pereira 9:20am Session 1: Data Needs

Moderator: Sharmila Murthy The Importance of Data in Helping Decision-Makers to Meet Current and Future Food,

Energy, Water (FEW) Needs by Deniz Susar (10-12 mins) This presentation will look at the role and the importance of data in achieving development goals particularly in engaging citizens in the development process. After giving an overview of emerging trends in the area of data such as “big data”, “open government data”, the presentation will highlight major initiatives from the UN System and around the World such as UN Global Pulse, Open Government Partnership, UN Guidelines on Open Government Data related to the Water, Energy and Food Security Nexus.

Open Discussion 10:10am Session 2: Understanding the role of ICT for development

Moderator: Patricia Guardabassi Major Trends and Challenges of ICTD by Revi Sterling (10-12 mins)

26

This talk will address the challenges of ICTD, covering topics such as scale, “utility gaps,” human motivation, replication of power structures, lack of integration, legal v. illegal tech, as well as other things that people often forget about when they “add tech and stir.” It will also consider how ICT can best serve development going forward, addressing issues such as white space/spectrum, pre-development readiness and integration with engineering for development.

Open Discussion 11am Coffee Break 11:15 Session 3: Promoting Innovation

Moderator: Laura Pereira Improving Water & Sanitation: The Role of Hackathons by Dan Shemie (5-7 mins) The CommCare Platform: The Role of Open Source by Nnenna Okoye (5-7 mins) Open Discussion

12:00 Lunch AND Group Photo

12:45pm Session 4: FEW Nexus with Climate Change

Moderator: Eunjee Lee

ICT and Food Vulnerability Analysis by Patrick Vinck (5-7 mins) ICT in Trans-boundary Water Resource Management by Yohannes Gebretsadik (5-7 mins)

Open Discussion 1:30pm Session 5: Perspective from Government Decision-Makers

Benin: Césaire Ahanhanzo (5-7 mins) Open Discussion 2:00pm Coffee Break 2:15pm Session 6: Key themes and Questions Break-out Groups

The break-out groups will be designed to foster a more in-depth conversation around the key themes and questions that emerge at the workshop. The groups will be pre-determined so as to promote discussion across sectors and disciplines.

3:45pm Plenary: Key Findings from Breakout Group 4:30pm Concluding Themes and Next Steps 5pm Adjourn

C. List of Questions Used to Form the Break-out Groups GROUP 1: Involving decision-makers:

27

Themes: Incentives Service delivery Power Questions: How can we create incentives for governments to use and provide data that could potentially threaten its power? How can we encourage policy-makers to share more data related to FEW needs? Why should I share data? How can we improve decision-making based on big data analysis on FEW issues? How to raise awareness about “virtual water imports”? How to bring the scientific method into public policy discourse? GROUP 2: Theory meets practice Themes: Quality Evaluation Response vs Awareness Trade-offs Questions: How to bring the scientific method into public policy discourse? How can we raise awareness about the trade-off data on FEW issues? Even in initiatives that cannot generate “real” change, isn’t there a role for awareness? Pitfall of Big Data and empirical thinking -> quality of Data, institutes to use it Complex Adaptive Systems (CAS) New systems develop from the bottom up -> path to development Evaluation of technology How to make ICT software end-to-end and get the necessary response? Praxis- how can theory guide successful use of ICTs? Data Quality challenges What to do about the trade-offs between lack of data and inaccurate data? Is there a need for standard evaluation of ICTD pilots? GROUP 3: Context Themes: Mindsets Scale Reality (Il)literacy Questions: Assume policy/decision-makers stay the same- isn’t it the responsibility of scientists to improve their message? Multi-scalar: Co-ordinating the analysis of levels and scales (more than global-local) Flexibility and digital data collection Language has to be communities’ Power of multimedia

28

Local community driven technology ICTs and scarcity of power resources, high illiteracy- How to adapt them to these realities? More data = better decision? Changing engineering mindset? Could society work together? Global made up of the local- How do we provide context for both? (and support both?) Data on “virtual water imports” GROUP 4: Institutions and Ethics Themes: Transparency Failures Capacity Questions: How do we create feedback loops that include the community we are trying to help? Should we talk @ ICT for ‘solving problems’ or ICT for creating dialogue and shared value? What is the role of “network brokers” in getting data for ICT and negotiationg transparency issues? When the mortality rates are known to the outside world, would that be able to empower communities and put pressure on governments? Hackathon as goal for empowerment and collective action Are technologies stealing livelihoods? Level of centralization for different applications? What does ICTD look like in rural development? Benefits vs costs of transparency?

29

IX. References Aker, Jenny C., and Isaac M. Mbiti. 2010. “Mobile Phones and Economic Development in Africa.” SSRN

eLibrary. http://ssrn.com.ezp-prod1.hul.harvard.edu/paper=1693963. Avgerou, Chrisanthi. 2010. “Discourses on ICT and Development.” Information Technologies and

International Development. http://itidjournal.org/itid/index. Beddington, John. 2009. “World Faces ‘Perfect Storm’ of Problems by 2030, Chief Scientist to Warn.”

The Guardian. March 18. http://www.guardian.co.uk/science/2009/mar/18/perfect-storm-john-beddington-energy-food-climate.

Cash, David W., William C. Clark, Frank Alcock, Nancy M. Dickson, Noelle Eckley, David H. Guston, Jill Jäger, and Ronald B. Mitchell. 2003. “Knowledge Systems for Sustainable Development.” Proceedings of the National Academy of Sciences 100 (14) (July 8): 8086–8091. doi:10.1073/pnas.1231332100.

Ceccarelli, S., and S. Grando. 2007. “Decentralized-participatory Plant Breeding: An Example of Demand Driven Research.” Euphytica. On-line version accessed March 14, 2007.

Cecchini, Simone, and Christopher Scott. 2003. “Can Information and Communications Technology Applications Contribute to Poverty Reduction? Lessons from Rural India.” Information Technology for Development 10 (2): 73–84. doi:10.1002/itdj.1590100203.

Corburn, J. 2005. “Street Science: Characterizing Local Knowledge.” In Street Science: Community Knowledge and Environmental Health Justice, 47–77. Cambridge, MA: MIT Press.

Donner, Jonathan, and Marcela X. Escobari. 2010. “A Review of Evidence on Mobile Use by Micro and Small Enterprises in Developing Countries.” Journal of International Development 22 (5): 641–658. doi:10.1002/jid.1717.

Etzo, Sebastiana, and Guy Collender. 2010. “The Mobile Phone ‘Revolution’ in Africa: Rhetoric or Reality?” African Affairs 109 (437) (October 1): 659–668. doi:10.1093/afraf/adq045.

Heeks, Richard. 2008. “ICT4D 2.0: The Next Phase of Applying ICT for International Development.” Computer 41 (6) (June): 26–33. doi:10.1109/MC.2008.192.

———. 2010. “Do Information and Communication Technologies (ICTs) Contribute to Development?” Journal of International Development 22 (5): 625–640. doi:10.1002/jid.1716.

Jasanoff, Sheila, and Marybeth Martello, ed. 2004. Earthly Politics: Local and Global in Environmental Governance. Cambridge, MA: MIT Press.

Kalil, Thomas. 2009. “Harnessing the Mobile Revolution.” Innovations: Technology, Governance, Globalization 4 (1) (January 1): 9–23. doi:10.1162/itgg.2009.4.1.9.

King, G. 1995. “Replication, Replication.” Political Science and Politics 28: 443–452. King, G. 1997. A Solution to the Ecological Inference Problem: Reconstructing Individual Behavior from

Aggregate Data. Princeton: Princeton University Press. Kuriyan, Renee, Isha Ray, and Kentaro Toyama. 2008. “Information and Communication Technologies

for Development: The Bottom of the Pyramid Model in Practice.” The Information Society 24 (2): 93–104. doi:10.1080/01972240701883948.

March, James, and Johan P. Olsen. 2004. “The Logic of Appropriateness.” ARENA Working Papers WP 04/09. Centre for European Studies, University of Oslo.

Maumbe, Blessing, and Julius Okello. 2010. “Uses of Information and Communication Technology (ICT) in Agriculture and Rural Development in Sub-Saharan Africa.” International Journal of ICT Research and Development in Africa 1 (1): 1–22. doi:10.4018/jictrda.2010010101.

North, Douglass C. 1987. Institutions, Transaction Costs and Economic Growth. Economic Inquiry 25 (3): 419–428.

North, Douglass C. 1990. Institutions, Institutional Change and Economic Performance. Cambridge, UK: Cambridge University Press.

Pfaffenberger, Bryan. (1988). “Fetishised Objects and Humanised Nature: Towards an Anthropology of Technology.” Man, New Series 23 (2): 236–252.

30

Pereira, Ângela Guimarães, Jean-Daniel Rinaudo, Paul Jeffrey, João Blasques, Serafin Corral Quintana, Nathalie Courtois, Silvio Funtowicz, and Vincent Petit. 2003. “ICT Tools to Support Public Participation in Water Resources Governance & Planning: Experiences from the Design and Testing of a Multi-Media Platform.” Journal of Environmental Assessment Policy and Management 05 (03) (September): 395–420. doi:10.1142/S1464333203001383.

Sachs, Jeffrey. 2008. “The Digital War on Poverty.” The Guardian. August 21. http://www.guardian.co.uk/commentisfree/2008/aug/21/digitalmedia.mobilephones.

Schmidt, Eric, and Jared Cohen. 2013. The New Digital Age: Reshaping the Future of People, Nations and Business.

Spiller, P., and Vogelsang, I. (1997). “The Institutional Foundations of Regulatory Commitment in the UK: The Case of Telecommunications.” Journal of Institutional and Theoretical Economics 153 (4): 607–629.

Unwin, P. T. H. 2009. ICT4D : Information and Communication Technology for Development. Cambridge Learning. Cambridge, MA: Cambridge University Press.

Wheeler, Deborah. 2011. “Does the Internet Empower? A Look at the Internet and International Development.” In The Handbook of Internet Studies, edited by Mia Consalvo and Charles Ess, 188–211. Oxford, UK: John Wiley & Sons.