1029 Vermont Ave NW, Suite 600, Washington DC 20005 USA

Phone: +1 202.449.4100 Fax: +1 866.681.8047 www.globalintegrity.org

Information Access in Health and Education Service Delivery: Report on the Ukraine and Kenya

February 2012

Raymond June and Abhinav Bahl

I. Introduction This report outlines the key findings of Global Integrity’s ongoing assessment of information access in health care and education service delivery in Kenya and the Ukraine. We conclude that these indicators can be applied across countries for benchmarking purposes in combination with country-specific indicators (in our parlance, what we call “core” and “non-core” indicators respectively) if carefully designed. A soupçon of the study’s origins highlights the theoretical underpinnings of these indicators, and their potential to be applied universally. At the behest of the World Bank’s Human Development Chief Economist’s Office (HDNCE), Global Integrity’s charge was to investigate the role that information access plays in the education and health sectors. Taking Macedonia as a starting point, we wanted to examine and interrogate normative assumptions about information transparency, which has been:

[…] theorized to be one of the key pillars of good governance in health and education service delivery, especially with respect to its potential to empower citizens to demand better service delivery and hold providers accountable. To unpack this claim, Global Integrity embarked on a pilot study of Macedonia in collaboration with a local partner organization by identifying and carrying out fieldwork on four key dimensions of information access in the health and education sectors that have been hypothesized to have a significant impact on citizen empowerment and participation in health and education service delivery decision-making and social monitoring in lower-income countries […]: 1) Basic Issues around the Existence and Usability of Information in Health Care and Education; 2) Redress Mechanisms that Enforce Accountability in the Health and Education Contexts; 3) Availability of Fiscal/Budget Information with Which to Conduct Citizen Audits of Local Schools and Clinics; and 4) Citizen Participation in Local Decision-Making as Influenced by Availability of Information (Global Integrity 2011).

Results from the Macedonia study challenged us to modify this theory of change – from questions about why certain information transparency and accountability mechanisms empower beneficiaries (or not) to what information-related institutions and mechanisms exist and how effective they are (or not). It served as a useful reminder once again that “empirical data can tell us what is happening far more readily than they can tell us why it is happening” (Lieberson 1987, quoted in Rosga and Satterthwaite 2009: 297). Therefore, a new conceptual vocabulary was needed to problematize normative assumptions of causal inferences.

2

Our assessments of these same sectors in the Ukraine and Kenya have strengthened our earlier contention that these indicators, while continuing to provide indeterminate evidence about the role of information transparency in strengthening citizen demand for better governance in social service sectors, can nevertheless be useful tools in cataloguing the existence and effectiveness of information-related accountability mechanisms and institutions. If questions are constructed with this specific focus in mind, then, we argue that a core set of universal indicators can be created and used for cross-country comparative purposes supplemented by a cluster of contextual indicators. In the remainder of this report, we first provide an outline of our indicator design and local fieldwork partner selection followed by a brief discussion of our methodology. Then we present our tentative country findings for Kenya and Ukraine. We conclude with a discussion of core and non-core indicators. II. Indicator Design and Local Fieldwork Partner Selection Indicator Set Modifications: From the onset it was evident that the indicator set developed for Macedonia would be insufficient in its current form for deployment in the field. Drawing upon the extensive work completed for the Macedonia pilot study, Global Integrity sought to modify the indicators with the intention of testing their applicability in locations as diverse as Kenya and the Ukraine. With the assistance of our colleagues in the World Bank’s HDNCE, the indicator set drafted for this study was developed through an iterative and collaborative process that involved interviews with health and education service delivery experts, discussions with local research partner organizations, and drawing on Global Integrity’s expertise in designing and publishing more than 100,000 governance indicators. A series of discussions with health and education public service delivery experts from the World Bank’s Washington DC and Kenya offices provided crucial feedback that was used to modify the indicators and create new ones to fill gaps learnt from in the Macedonia assessment. The feedback received centered on organizing and categorizing the indicators to enhance their portability for ease of deployment in the field given the volume of indicators involved. The World Bank stressed the need to make the unwieldy indicator set more manageable by organizing the indicators into thematic clusters to facilitate the research process, which was recognized as largely reliant on interviews with administrators and beneficiaries in the field. The discussion was also beneficial in yielding ideas for new indicators. Consequently, the new indicators drafted focused on information-provisioning related to the locations of facilities, results of external monitoring exercises to enhance government policy, distribution and spending of donor-funded grants, maintenance of essential drug lists and stock out rates, and budget allocation and execution. The indicator sets were also vetted by the National Taxpayers Association (NTA) and Kiev International Institute of Sociology (KIIS), our local fieldwork partners. Discussions with

3

researchers from both local partner organizations provided crucial insight into the relevance of each indicator in their respective local country contexts. Moreover, the partners described country-specific information-provisioning standards, information-transmittal methods, and social accountability mechanisms that were useful in refining the indicator selection process and calibrating scoring criteria across the indicator set. Upon the incorporation of feedback received from our stakeholders, including refinements by Global Integrity staff over a period of months, the finalized indicator set was transmitted to our local partners and deployed in Kenya and Ukraine using Indaba, Global Integrity’s online distributed fieldwork platform. Selection of Local Research Partner Organizations: Global Integrity actively recruited potential research teams through referrals from the World Bank and other organizations it has worked with in the past to identify the appropriate local research partner organization to conduct this pilot study. After a competitive internal selection process, we decided to team up with the National Tax Payers Association in Kenya and KIIS in Ukraine based on several factors, including, but not limited to: the formal qualifications of local teams of experts; sufficient capacity to conduct distributed field research: prior experience in evaluating service delivery in the health and education sectors: expertise in social accountability and service delivery monitoring mechanisms; as well as available resources. III. Fieldwork Methodology Like Global Integrity’s other national-level and sub-national work, this sector-level study favored an expert assessment methodology over alternative approaches such as household/ beneficiary surveys, citizen scorecards, or public expenditure tracking surveys. As in the Macedonia pilot study, we employed two types of indicators: “in law” and “in practice.” All indicators were anchored by unique scoring criteria to promote inter-coder reliability and supported by detailed referencing and narrative commentary. Further, the indicators, regardless of type, were scored on the same ordinal scale of 0 to 100 with zero being the worst possible score and 100 perfect. “In law” indicators provided an objective assessment of whether certain legal codes, fundamental rights, types of information, government institutions, and regulations existed. These “de jure” indicators were scored with a simple “yes” or “no,” with “yes” receiving a 100 score and “no” receiving a zero. “In practice” indicators addressed the “de facto” issues such as implementation, effectiveness, enforcement, and citizen access to those same “de jure” mechanisms and institutions. As scoring those indicators usually requires a more nuanced assessment, these “in practice” indicators were scored along an ordinal scale of zero to 100 with possible scores offered at 0, 25, 50, 75, and 100. In Kenya the scoring of in law indicators relied on detailed desk research, which involved a review of the country’s laws and regulations relating to information access in the health and

4

education sectors. NTA researchers reviewed the constitution of Kenya, relevant acts and statutes, ministry service charters, legal notices and circulars, among other documents that constitute the country’s statutory regime governing health and education service delivery. The information gathered from this legal review was used to score all in law indicators. Data for scoring in practice indicators were largely sourced through extensive field interviews. For the health assessment, interviews were conducted across 30 health centers and dispensaries in Kisumu and Nairobi. Interviewees consisted of healthcare beneficiaries including patients, as well as nurses, doctors, district medical officers, and hospital administrators. To score education indicators, interviews were conducted across 20 primary schools in Nairobi and 21 primary schools in Kisumu. Principal interviewees consisted of service beneficiaries, i.e. students and parents, as well as head teachers, district and municipal education officers, Ministry of Education officials, and representatives from parent and teacher organizations. Also interviewed were members of various civil society organizations such as FIDA Kenya, the Kenya National Union of Teachers, and the Elimu Yetu coalition. Interviews conducted by researchers from KIIS constituted the main source of data for the health and education assessment in the Ukraine, supplemented by desk research. This included discussions with hospital, public health, and school officials, physicians, patients, teachers, parents, journalists, and NGO staff. Attuned to regional diversity and linguistic differences, the researchers selected two historical regions that represented a large cross-section of the population: the rural and predominantly Ukrainian-speaking Galichina region in the west (including Ternopilska, Lvivska, Ivano-Frankivska, and parts of Chernivetska oblasts), and the more urbanized/industrial and majority Russian-speaking Donbass region in the east (including Donetska, Luganska, and parts of Kharkivska and Dnipropetrovska oblasts). Fieldwork was also conducted separately in Kiev because it is the country’s most densely populated city, relatively advanced economically, and has a significant concentration of expert respondents; results from Kiev were folded into the Galichina scorecard. IV. Findings for Kenya and Ukraine Kenya: Data from Kenya reveal a persistently weak legal infrastructure across both health and education sectors. Despite the existence of a raft of laws, regulations, and legal notices that govern the delivery of public services in health and education, deficiencies in the legal regime, particularly relating to information access, persist. While the legal framework governing information access in healthcare appears to be more robust, weaknesses in the education sector are particularly pronounced. The study shows that the extant legal and institutional architecture does not comprehensively set forth the demand- and supply-side institutions and mechanisms of information access theorized to enhance accountability in the delivery of public health and education services. Instead, an assemblage of fragmented legal statutes governs the availability of information in the delivery of health and education services. Provisions concerning the availability and quality of information, social accountability mechanisms, and citizen input into education policy are absent in the Education Act, the primary

5

legal instrument that defines national guidelines on the establishment and administration of education institutions, development of curricula, and training of teachers. The Act does not contain requirements for the government to make available information on school locations, meals, curricula, teacher attendance, student disciplinary action, and administrator codes of conduct [indicators 11.1, 11.6, 11.12 – 11.15, 11.18, 11.28]. Unsurprisingly, legal avenues for citizens to deliver complaints to service delivery agencies on issues pertaining to education service delivery such as overcrowding in classrooms, fees, student rights violations, curriculum, meals, and teacher absenteeism are simply nonexistent [indicators 12.1, 12.8, 12.15, 12.22, 12.29, 12.36, 12.43, 12.56]. Information dissemination requirements on formal and informal citizen participatory mechanisms for input into the policymaking process are also absent in law. Amidst these gaps, bright spots include the legal requirements for public availability of education budget information [sub-categories 3.1 – 3.6]. Laws that make public budget allocations are robust at the national and district levels. However a disjuncture is apparent at the local level, where individual school budget disclosures are not mandatory [indicator 13.15]. The health sector paints a more complicated portrait of information transparency given the skein of intersecting laws and regulations—the Public Health Act, Medical Practitioners and Dentists Act, Pharmacy and Poisons Act, Dangerous Drugs Act, among others—that govern the delivery of health services in Kenya. Budget information availability to track resource allocations and flows is statutorily opaque [sub-categories 3.2 – 3.6]. For instance, the study revealed that recurrent and capital budgets, revenue and expenditure reports, donor allocations, resource transfer mechanisms, as well as audit findings are not mandated to be made public at the local health facility level [indicators 13.3, 13.6, 13.9, 13.12, 13.14, 13.16, 13.18, 13.21, 13.24]. In contrast, legal requirements for information on formal and informal citizen participatory mechanisms, including patient advocacy groups and consultative forums, are more robust. [indicators 14.1, 14.4, 14.5, 14.9, 14.12, 13.20]. The Community Strategy Implementation Guide published by the Ministry of Health spells out the linkage structures at district, divisional, health facility and community levels that are expected to provide citizens with sufficient representation in all issues affecting service provision [indicators 14.4, 14.5, 14.12]. With weak legal regimes governing information access in the two sectors, concomitant deficits in the de facto implementation and effectiveness of information transparency and accountability institutions and mechanisms are to be expected. However, contrarily, the data reveal a different and interesting picture. It appears that in practice citizens seek information on many aspects of health and education service delivery through alternate mechanisms despite the absence of a statutory ruling mandating that information be made accessible. Low “In law” scores and higher “In practice” scores resulting in a negative “implementation gap” is a manifestation of this phenomenon. Given the methodological limitations of this assessment, the reasons for accessing this information are indeterminate—i.e., we cannot determine why citizens are accessing this information, but we can determine the modalities of access—i.e., how citizens are accessing this information. Thus, in the education sector, while information availability requirements for records of overall school performance measurements, disciplinary action taken against students, teacher attendance, and school curricula are not codified in law, the study reveals that such information is still sought by citizens [in practice indicator sub-categories 1.1 – 1.8]. More

6

specifically, even though a legal requirement to make public school performance records does not exist, head-teachers and parents interviewed for this study note that the information is disseminated through parent-teacher meetings, flyers on notice boards, results slips provided to students, and report forms issued to pupils [indicator 11.29, 11.30]. Likewise, information about the school curriculum is available in the head teacher’s office, city education offices, local bookshops, and upon requests from the Kenya Institute of Education [indicators 11.10, 11.11]. In the delivery of health services, this phenomenon is less pronounced, occurring only in a few instances. For example, there is no express provision in law to make information on a code of ethics for healthcare professionals publicly available; yet service providers in the course of service delivery occasionally disseminate relevant information on codes of conduct to beneficiaries [indicators 11.20]. It would appear that information access mechanisms and institutions in health and education service delivery are effective not because of the presence of robust legal frameworks but in spite of their absence. A significant exception is healthcare delivery in Nairobi where the general legal framework of information access is more robust than its implementation resulting in a manifest implementation gap. The selection of two distinct locations to field the indicators—Urban Nairobi versus rural Kisumu—was by design as part of a larger and deliberate attempt to tease out and measure the differences in de jure institutions and mechanisms related to information access. Regional variances in the legal regime do not occur in Kenya—the laws governing information availability of public health and education services in Kisumu are the same in Nairobi. In practice, while minor variations in information access are noted between rural and urban regions, the discrepancies are not significant in the case of education service delivery. This is hypothesized to be the result of Kenya’s centralized legal infrastructure, which enacted at the national level, has a normalizing effect across regions. Individual local practices at education institutions and health facilities seem to coalesce around national guidelines delineated by the Ministry of Education, Ministry of Public Health and Sanitation, as well as the Ministry of Medical services. In the health sector the differences in information availability between rural and urban health facilities are cast into sharp relief. An implementation gap in Nairobi is evidence that the de facto implementation of information provisioning mechanisms significantly lags behind the standards set out in law. By contrast Kisumu fares better in all categories but one (Citizen Participation and Decision-making) despite being subject to the same legal regime. The data show that citizens in Kisumu in practice access information through various alternate mechanisms that are not codified and established in law.1 Ukraine: Data from the Ukraine education and health care scorecards can be distilled to five thematic frames: significant implementation gaps, a supine legal regime, sectoral differences, regional variation (or lack thereof), and top/bottom performances in the area of accountability and transparency. Perhaps the most graphic finding is the implementation gap in both sectors, i.e., the disjuncture between legislation and its implementation and enforcement, a problem thematized in

                                                                                                               1  This section is subject to revision.  

7

our Macedonia report as well. We can identify a few areas where this chasm is particularly pronounced. For instance, existing legal provisions on the public disclosure of health care budgetary information are attenuated in practice by byzantine institutional arrangements and even outright threats. Citizens have the legal right to access information on local health clinics’ acceptance of private donations, although the law is vague about what kind of budget information should be provided and because of this government, local authorities, and clinic representatives provide information that is difficult to compare and analyze. In the Galichina region, consequently, most donations channeled through charities or third parties are in practice very hard or impossible to track (or control) for government representatives and citizens, making it difficult to know whether donations actually reach hospitals. In Donbass, this problem seems to be exacerbated by a lack of demand/public awareness among citizens for such information [indicator 13.13]. In addition, while there is a legal framework in place for information to be disseminated on health care service standards (ranging from wait times to user fees), the public spread of this information in both regions is again limited by the public’s lack of demand/awareness for such information; or, when information is provided, it is considered too technical for easy comprehension by average citizens [indicator 11.32]. Furthermore, there are weaknesses in the legal framework itself particularly in the health sector. To be sure, an absence of key legislation in Donbass and Galichina contributes to attenuating the law on the books – the lack of information on formal redress mechanisms to complain about issues such as violations of patient rights in health, for example [e.g., indicators 12.1, 12.7, 12.13]. But just as significant are some of the legal loopholes that have disabled the effectiveness of existing legal provisions. This is underscored in the “Law of Ukraine on Public Associations” that determines the procedures for forming associations of citizens to satisfy and protect their social, economic, national, cultural and other common interests, but does not require such information to be shared and disseminated publicly [indicator 14.1 Education]. Similarly, while there is legislation mandating information be made available on how citizens can lodge complaints on issues like absenteeism, the language is vague as there is no specific provision for submitting complaints about health and medical services [e.g., indicator 12.25]. These findings corroborate the recent National Integrity System study of the Ukraine by Transparency International, which concludes that “outdated legislation on citizen access to public information” among other issues has led to an “inadequate legal framework” and weak institutions of public integrity (Chebanenko et al. 2011: 21-22). As alluded to in the previous paragraph, it is also evident that there are sectoral differences in the Ukraine. Although several areas still need significant improvement, information transparency in education service delivery is nevertheless more robust than its health counterpart. The overall legal framework for education in both regions is stronger than health, and this tends to hold true when the indicators are disaggregated to a more granular level. Citizens, for example, have more legal and practical recourse to registering their grievances in matters related to education. Information on how to file complaints on various issues is disseminated by civic education projects promoted by NGOs and the media [e.g., indicators 12.16 and 12.23]; records of complaints are kept in the City Council and District/City Department of Education although gathering these records may take time [e.g., indicator 12.20]; and disciplinary measures are applied, though not always consistently [indicator 12.50].

8

In practice, there are official consequences, disciplinary measures, and rewards applied based on the monitoring of record-keeping practices at school. (Indicator 12.50 Education, Donbass.) Inspections are conducted regularly and, as a rule, disciplinary actions are taken when violations are revealed. Most often these are fines, reprimands; less often – expulsion of school administration.

In contrast, there are no legal provisions that specifically encompass mechanisms that address school-related complaints [e.g., indicators 12.1, 12.7, 12.13]. While information is sometimes disseminated by NGOs and made available on the websites of local administrations, the complaints are often undifferentiated and difficult to record, thereby making analysis and possible follow-on recommendations difficult [indicators 12.8, 12.37]. In addition, there is more space for citizens to participate in decision-making related to education, though this is still limited to providing schools with resources rather than influencing policy-making [see indicator 14.3 Education, Galichina]. Parents’ associations/committees in each school allow parents to make some recommendations to school management (on school meals, for instance), and such information is widely disseminated through parents meeting, school bulletin boards, and the media [indicator 14.2]. In contrast, information on citizens’ right to participate in health care and service delivery decision-making are made public, but there is no requirement for this information to be disseminated. Citizens also have the right to join citizen patient advocacy groups, but de facto they have little awareness of the existence of such community forums and advocacy groups [e.g., indicators 14.1, 14.3, 14.9]. While there are some notable variations in the performance of sectors, however, it is worth mentioning that significant polarities between regions have not been discerned in Global Integrity’s assessment. For while Galichina and Donbass can be differentiated along historical, socio-cultural, demographic, and political lines, these distinctions do not manifest themselves clearly in the country’s public integrity system vis-à-vis information transparency in sector service delivery. One possible explanation is that much of the legal framework in place seems centralized at the national rather than sub-national/local level, thereby creating a potentially homogenizing effect. But even de facto differences appear relatively minuscule. Finally, what are the best and worst performances in the more than two-dozen sub-categories of integrity assessed in each sector? Data from Global Integrity’s scorecards reveal government monitoring in health and education to have the most robust regulations and enforcement, while information on citizen participation in health care decision-making is the most poorly rated issue assessed (indeed, few experts and patients interviewed had even heard of such participatory mechanisms). In health, regular state oversight inspections – which are limited in their goals; for example, they evaluate the general level of patient care, but not treatment effectiveness – to ensure medical institutions are complying with standards are mandated by law. However, the information on the results of such exercises is unevenly disseminated [e.g., indicators 12.48, 12.49, 12.54]. By way of contrast, direct citizen engagement in health-related policies via community/patient advocacy groups as well as advisory boards is uncommon, and thus information on such rights is rarely circulated publicly [e.g., indicators 14.7, 14.13].

9

In practice, information about citizen/CSOs’ basic rights to participate in citizen oversight boards is disseminated to the public. (Indicator 14.7 Health, Galichina.) Experts, expect one, have never head of the existence of such practices. “Activists are not allowed in such councils if they exist. Often the supervisory councils are formed on the principle of sponsorship of the hospital. They are created for status sake, and not for control. They have no dissatisfied patients. Maybe there is a nominal representative of a pocket NGO. Public council at ‘OHMATDET’ National Children Hospital is the only positive example I know.” ~ Viktoria Tymoshevska, Public Health Program Director, International Renaissance Foundation, October 5, Kiev

(In education, state monitoring of schools is a standard practice where “results of government inspections and evaluations of the quality of work of schools are presented to citizens in an understandable and clear way” [indicator 12.62]. However, the practice of publicizing the results of regular state school inspections is not fully developed and may vary from school to school and city to city [see indicator 12.62 Galichina].) There are other lines of inquiry that undoubtedly lie dormant in this schematic analytic exercise, but that can (and should) be raised to the surface in further country-specific studies. For now, we would simply like to reiterate a plea made earlier in our report on Macedonia for the importance of complementary qualitative tools; for grounded perspectives that place health/education service delivery and the role that information transparency plays in those sectors in context; for analyses sensitive to social process and relations (Global Integrity 2011: 15-16). If these pilot indicators can be used to map and catalog accountability/transparency institutions and their effectiveness, then the question of why and how these institutional mechanisms have come to be require a different set of analytics drawn from political economic and anthropological/sociological approaches. Take, for instance, public health, which has emerged as a central site in the Ukraine for citizens to demand better governance – indeed, to make citizenship claims more broadly – in the aftermath of Chernobyl (Petryna 2002). In what ways has the legacy of this disaster shaped the institutional contours of Ukraine’s health system, the provision of information, and the ability of public health providers to serve the needs of its citizens? In turn, how has the shadow of Chernobyl opened up and closed off the ability of citizens to hold the state accountable via access to information (among other practices)?2 These types of questions and attendant issues

                                                                                                               2 While the objective of this particular project was not to examine the effects of Chernobyl, a separate KIIS project conducted for UNDP demonstrated that people who lived in the catastrophe zone (sufferers and non-sufferers) were even less pro-active in demanding good governance from the state. The larger point here is that historical-cultural “legacies” exert a structuring, though not necessarily deterministic, effect into the present that can and should be examined and unpacked in complementary studies.

10

can only be addressed if indicator-generated data are combined with contextually “thick” analysis. Understanding the wider socio-political and historical contexts, and complexities, of specific social sectors in specific countries will go a long way towards creating more effective and informed policy choices. Having drawn attention to some country-specific details that this assessment has helped to unravel, we move next to a discussion of whether these indicators can be applied universally. We answer yes, but with caveats. V. Core and Non-Core Indicators Along with cataloguing the information-related accountability mechanisms and institutions in the health and education sectors, an objective of this assessment is to test the feasibility of a universal set of indicators to be deployed across geographies for comparative analyses. The postulated theory of change underpinning this study is that information provisioning in health and education service delivery empowers citizens, strengthening the demand-lead pull towards more accountable and efficient public service delivery. Results from the pilot assessment in Macedonia to investigate that hypothesis proved that an indicator-based expert assessment methodology is useful in cataloguing information-related institutions and mechanisms rather than explaining the causal mechanisms behind citizen empowerment and service delivery improvement (i.e., the assessment answered the "what" question more effectively than the "why" or "how" questions). Following this line of inquiry, our next charge was to demonstrate the applicability of such an assessment in other jurisdictions by designing and deploying a similar set of indicators across geographies to enable cross-country comparative analyses while effectively capturing local nuances. Or to put the question in another form: Can an indicator set be developed to contain both “Core” indicators (universal in their applicability that travel across jurisdictional boundaries) as well as “Non-core” indicators (supplemental location-specific questions that capture local specificities for the purposes of context-specific analysis)? The assessments in the health and education sectors in Kenya and Ukraine allowed us to answer this question by testing the portability and replicability of our indicator set. Our conclusion is that the indicator set can be replicated and similar assessments can be reproduced across countries albeit with an important qualification: the role of human judgment and authority should be rendered explicit, not only when designing and selecting core and non-core indicators (with participation from all parties involved), but also in thinking through who will adapt and use the indicators. This section contains a fuller discussion of the rationale for the classification of core and noncore indicators based on lessons learned from conducting assessments in Kenya and the Ukraine. We have observed through our assessments that non-core indicators occur by exception since the majority of indicators appear to be universal.

11

Education Non-core Indicators: Types of Information/ Complaints by Issues - Sub-Categories 1.1, 2.1 - 2.6, Indicators 11.2, 11.3, 11.6, 11.7, 11.8, 12.29, 12.30, 12.31, 12.32, 12.33, 12.34, 12.35 - Indicators on complaint mechanisms disaggregated by issue are neither applicable nor considered a priority across local contexts, warranting a non-core classification. For instance, in Kenya not all countries/jurisdictions institute school meal programs and hence the question of complaints on school meals is moot. Similarly, teacher absenteeism is not a relevant issue in the education system in Ukraine. Researchers also noted that information on the relative distances of schools is not relevant and rarely, if ever, requested. Forms of Citizen/ Community Participation - Sub-categories 4.1, 4.3, Indicators 14.5, 14.6, 14.13, 14.18, - Participation in community decision-making through formal and informal means is contextual since the forms and modalities of participation are socially and culturally determined. In the context of education, for example, information on citizens’ ability to participate in informal community networks and forums for school decision-making was identified as irrelevant in Ukraine. Further, there is no recognized practice concerning the establishment of school monitoring committees in Ukraine. Disaggregation of Information Access Issues by Gender/Income/Urban-Rural settings - Sub-category 4.4, Indicators 14.17, 14.18, 14.20, 14.21, 14.22, 14.23, 14.24, 14.25 - The assessment showed that issues of gender, income, and rural/urban settings are not universal frames of reference neither across geographies nor cultures. Unsurprisingly, these characteristics are highly context-specific and hence indicators designed to disaggregate data along these lines should be approached with an appropriate measure of caution, attentiveness, and discretion. For example, in Galichina, Ukraine gender is not considered the discriminating factor in participatory decision-making related to school supplies, meals, and curriculum. As a result, the disaggregation of data by gender was deemed irrelevant by researchers for the issues mentioned.

Education Core Indicators: Access to Information Legal Framework - Category 1 - Data from Macedonia, Ukraine, and Kenya demonstrate that indicators assessing the existence of relevant laws, statutes, and regulations are relevant and replicable across country contexts and can be considered universal core indicators. Budget Information: - Category 3

12

- Indicators focused on information related to budgetary flows, policies, rules, and institutions are all relevant to Kenya, Ukraine, and Macedonia although the level and quality of information provided varies. Making such information open is widely considered a “best practice” because the control of the government budget is a cornerstone of good governance. Thus, these indicators can be considered universal. Health Non-core Indicators: Formal Complaint Mechanisms: - Sub-categories 2.1-2.7 - These mechanisms vary from one local context to another and involve a wide-range of institutional bodies ranging from Ombudsmen to auditing agencies. For instance, in the Ukraine alone, citizens can submit their complaints to media, state authorities, local governments, and citizen associations. Moreover, which institutional mechanism handles medical complaints is not specified. Hence, crafting contextual indicators on formal complaint mechanisms is recommended. Disaggregation of Access to Information Issues by Race/Gender/Income/Region: - Sub-category 4.5 (indicators 14.15-14.18, 14.19, 14.20) - Preliminary data from indicators demonstrate that instances in which special legal provisions are made according to gender, race, income, and regional differences are rare. Furthermore, the level of disaggregated data at the de facto level ranges tremendously, so that constructing these types of indicators on a country-specific basis is advisable. Citizen Auditing / Community Participation: - Sub-categories 4.1-4.5 (i.e., Category 4) - The participation of citizens in information-based health care service delivery is socially-culturally specific and therefore contextual. Citizen oversight boards, for instance, are largely a foreign concept, while formal and informal community participatory mechanisms vary so greatly across geographies that indicators on such mechanisms are probably best co-designed with local valences in mind. Complaints by Issues / Types of Information: - Sub-categories 2.1-2.6 - We recommend indicators organized by the types of information being accessed (or complaints by issues) such as wait times, user fees, etc. be non-core because not all issues are relevant, common, or pressing to the country (e.g., absenteeism in the Ukraine). Instead, the relevant indicator-by-issues should be identified in concert with the local partner who knows best what types of information are germane to the country. Health Core Indicators: Access to Information Legal Framework:

13

- Categories 1-4 (e.g., indicators 11.5, 12.13, 12.38, etc.) - Like the education sector, the suite of indicators centered on the legal framework in health is applicable and replicable across countries. Budget Information: - Category 3 - Indicators focused on information related to budgetary flows, policies, rules, and institutions are all relevant to Kenya, Ukraine, and Macedonia although the level and quality of information provided varies. Making such information open is widely considered a “best practice” because the control of the government budget is a cornerstone of good governance. Thus, these indicators can be considered universal. Informal Redress Mechanisms: - Category 2 (indicators 12.4, 12.10, 12.16, 12.22, 12.28, 12.34, 12.40, 12.41) - Citizens frequently make recourse to informal mechanisms to file complaints. In Kenya, for example, civic and religious leaders fulfill this role. In Macedonia, as we noted earlier, the media has served as an informal forum for citizen complaints. (These practices are less common in the Ukraine, however.)

Although we have identified many indicators that can be applied across countries, we are persuaded that caution must nevertheless be exercised when using them. More specifically, it matters who the actors selecting and using the indicators are, and staying mute on this issue will continue to cloud how universal/core indicators are deployed and interpreted (Rosga and Satterthwaite 2009). For example, officers from aid agencies or multilaterals may want to assess compliance factors and whether states have met certain legal standards around information-based service delivery, or if basic information on certain mechanisms of accountability is made available to citizens. However, CSOs may turn to these indicators to determine the effectiveness of such information for advocacy and outreach purposes (e.g., whether citizens actually make use of redress mechanisms). Being cognizant of who adapts the universal indicators to the country level (as well as comparatively), then, becomes critical because of divergent objectives and goals. Furthermore, as Rosga and Satterthwaite again remind us, multilateral/aid institutions are imbued with a certain authority/power that many CSOs or other actors don’t have, and this has implications for how core indicators are identified and used for monitoring and other purposes. The question of human judgment and authority must therefore always be kept at the fore, and not obscured from view (ibid.). Reflexivity should be a central concern for all involved.

14

References:

Chebanenko, Olena, Roman Golovenko, Dmytro Kotlyar, and Denys Kovryzhenko (2011). National Integrity System Assessment: Ukraine 2011 (TORO Creative Union/Transparency International Ukraine). Global Integrity (2011). “Information Access in Health and Education Service Delivery: Pilot Project Report on Macedonia.” Petryna, Adriana (2002). Life Exposed: Biological Citizens after Chernobyl (Princeton: Princeton University Press). Rosga, AnnJanette and Margaret L. Satterthwaite (2009). “The Trust in Indicators: Measuring Human Rights,” Berkeley Journal of International Law 27/2: 253-315.