1029 Vermont Ave NW, Suite 600, Washington DC 20005 USA

Phone: +1 202.449.4100 Fax: +1 866.681.8047 www.globalintegrity.org

Information Access in Health and Education Service Delivery: Pilot Project Report on Macedonia [Released January 2011]

Raymond June and Norah Mallaney

EXECUTIVE SUMMARY Information transparency is theorized to be one of the key pillars of good governance in health and education service delivery, especially with respect to its potential to empower citizens to demand better service delivery and hold providers accountable. To unpack this claim, Global Integrity embarked on a pilot study of Macedonia in collaboration with a local partner organization by identifying and carrying out fieldwork on four key dimensions of information access in the health and education sectors that have been hypothesized to have a significant impact on citizen empowerment and participation in health and education service delivery decision-making and social monitoring in lower-income countries. The indicators in this pilot study do not address why certain information transparency and accountability mechanisms may or may not empower beneficiaries, but they proved a useful tool in mapping and identifying what information-related rights, institutions, and mechanisms exist and how effective they are (or are not) at providing health- and education-related information to service beneficiaries. While the indicators do not provide a comprehensive roadmap for reform and intervention, they are useful tools to catalogue key local level information transparency and redress mechanisms vis-à-vis sector service delivery. When combined with other complementary analysis and data, these can be used to identify possible entry points for policy reform. Key findings from Macedonia include: significant implementation gaps exist between information access laws (as they relate to education and healthcare service delivery) and their implementation and enforcement; the absence of citizen audit and community monitoring groups in both sectors; the relative robustness, in contrast, of official government audits and monitoring of these sectors; the importance of informal mechanisms and processes in information sharing and dissemination as they relate to key HD sector service delivery; and the relevance of how information is transmitted. The report concludes that this pilot assessment tool merits scaling up in other countries, if certain caveats are kept in mind, by fielding a core set of universal indicators that can be used for cross-country analyses complemented by more bottom-up and locally-specific indicators and/or qualitative analysis on a one-off basis. This report includes the preliminary findings of a pilot exercise aimed to test the feasibility of collecting information on the policies and practices related to access to information in education and health. The survey was not intended to be representative or comprehensive.

2

I. INTRODUCTION In the past several years, there has been growing interest in exploring how human development outcomes can be improved by strengthening the performance and quality of delivery in major public service sectors such as education and health. In turn, effective service delivery is increasingly understood to be only as good as the quality of its governance, as noted by the World Bank (Fiszbein et al. 2009). As the relationship between good governance and service delivery has thickened, multi-lateral institutions, NGOs, academics, and policy makers are increasingly calling for numerical indicators that can effectively and systematically assess the quality of governance in service delivery in country-specific and cross-country contexts. In “Making Services Work,” experts from the World Bank’s HD Chief Economist’s Office (HDNCE) comprehensively make the case for more robust HD governance measures to facilitate and improve international/intranational benchmarking, impact evaluation for policy, and development project monitoring of service delivery. While they suggest training the analytic and practical lens on five components of service delivery that make up overall good governance performance – human resources, financing systems, critical inputs, information, and provider entry – they make evident that there is still much work to be done in terms of refining how and what to measure (ibid.). It is within this context that World Bank’s HDNCE approached Global Integrity to carry out a pilot assessment of information transparency (one of the five pillars theorized to contribute to effective service delivery in “Making Services Work”) in the health and education sectors in order to develop: 1) a set of actionable indicators that could identify whether service delivery beneficiaries are empowered by information (or not) to hold health and education providers accountable, 2) test these indicators in one lower-income country via collaboration with a local partner civil society organization (CSO), and 3) promote these indicators within the broader HD community. In consultation with the HDNCE team, Global Integrity agreed to focus narrowly on the issue of information access in developing the pilot indicators. Access to information has been theorized to contribute to citizen empowerment in service delivery, but to date much of the evidence has been anecdotal rather than systematically examined. One of our objectives with this fieldwork was to develop a tool that might be useful in practically unpacking the mechanics of that theory of change in a particular country context. To this end, and working closely with the HDNCE team, Global Integrity identified a core set of access to information indicators for this pilot project that are hypothesized to have a significant impact on citizen empowerment and participation in health and education service delivery decision-making and social monitoring in lower-income countries. The indicators (two unique sets) are clustered along the following key dimensions of information access to health and education service delivery: 1) Basic Issues around the Existence and Usability of Information in Healthcare and Education; 2) Redress Mechanisms that Enforce Accountability in the Health and Education Contexts; 3) Availability of Fiscal/Budget Information with Which to Conduct Citizen Audits of Local Schools and Clinics; and 4) Citizen Participation in Local Decision-Making as Influenced by Availability of Information.

3

It is important to make clear at the outset that in order to make this particular assessment tool meaningful, we had to delimit its scope. An implicit theory of change underpinned our desk research and the subsequent development of the resultant indicators: that improved access to key information might empower service beneficiaries (parents, students, patients) to demand improved service delivery and/or better hold their service providers accountable. To arrive at why certain information transparency and accountability mechanisms may or may not empower beneficiaries, however, deserves significantly deeper attention in subsequent fieldwork and research beyond this particular indicator exercise (see Section 4 for details). This pilot study of Macedonia focused, instead, on addressing prior questions related to what information-related institutions and accountability mechanisms exist and how effective they are (or are not). The stress on the what, rather than the why, made this assessment akin to an inventory of key sector-specific information transparency mechanisms and their effectiveness. We suggest bearing this distinction in mind when reading this report and the indicator data themselves. While the indicators do not provide a comprehensive roadmap for reform and intervention, they are useful tools to catalogue key local level information transparency and redress mechanisms vis-à-vis sector service delivery. When combined with other complementary analysis and data, these can be used to identify possible entry points for policy reform. In this report, Global Integrity and the Center for Research and Policy Making (CRPM), our local partner organization in Macedonia, critically examine the research process and preliminary findings of this pilot study. The report is structured as follows. The first section, Lessons Learned, describes how we designed the indicators, selected a CSO partner to field the data, and lessons learned from this process. In the second section, Macedonia Findings, we analyze the results and highlight a few key findings of local level information transparency and redress mechanisms that may contribute to and inform evidence-based governance policymaking. The concluding section, Future Directions, turns to a discussion of the efficacy and challenges associated with cross-country comparisons at the conceptual level, the importance of indicators being sensitive to unique local country conditions, and the possibility of scaling up this assessment to other countries. II. LESSONS LEARNED In order to help assess whether service delivery beneficiaries are empowered to hold providers accountable and demand improved services in health and education based on their access to information, Global Integrity developed a unique set of indicators to be applied to a yet-undetermined location. With the assistance of our colleagues in the World Bank’s HDNCE, Global Integrity employed a collaborative approach to develop this indicator set. Our process was broken into three stages: background research (both literature review and interview based), synthesizing and writing the indictors, and finally, soliciting feedback and refining the indicators based on comments from relevant stakeholders and external experts. Once the assessment tool was finalized, researchers at CRPM fielded the indicators. In this section, we reflect on the indicator design process, discuss the selection of the local partner organization, and briefly describe the fieldwork methodology.

4

Background Research: Global Integrity is well-versed in issues of transparency and accountability, having carried out fieldwork assessing the existence and effectiveness of anti-corruption systems in more than 100 countries and having published more than 80,000 quantitative governance and anti-corruption indicators in the process. However, as non-experts in the fields of healthcare and education, Global Integrity’s first step was to engage in an intense literature review. Our goal through this research was to better understand the complexities in the process for delivering healthcare and education to citizens and the common bottlenecks blocking beneficiaries’ receipt of quality services, especially those linked to information transparency. Some questions we sought to answer, with an eye toward addressing issues of “what” rather than “why” (as discussed above), included: what are the steps for healthcare or education resources (budgetary or otherwise) to reach the point of delivery? Who are the main actors in the processes of healthcare and education service delivery? Do sector-specific citizen redress mechanisms exist for either healthcare or education? Are there standards or “best practices” around information transparency in healthcare or in education? How closely are sector-specific information disclosure mandates linked with national-level freedom of information laws? What is the role of personal or community networks in the flow of sector-specific information? We also explored any existing international service standards for healthcare and education and the prevalence of information publishing requirements. We were interested in other research and fieldwork assessing the level of citizen uptake of available information in healthcare and education. For instance, what types of information do public or private service providers typically release? Do beneficiaries actually make use of the information available to them? What are the challenges or barriers to citizen use? To address these complex and interconnected questions, we completed extensive desk research, culling from sector-specific studies produced by international groups such as the World Bank, the World Health Organization, UNESCO and the Center for Global Development, to name a few. We also sought out research on corruption in service delivery produced by organizations such as Transparency International and U4 Anti-Corruption Resource Centre. In addition, we conducted interviews with various Washington-based sector-specific experts including Courtney Tolme at Results4Development; Dan Ritchie of the Partnership for Transparency Fund; and Halsey Rodgers, Paolo Belli, and Jishnu Das from the World Bank. Their comments and reading recommendations were extremely helpful in guiding us through the overwhelming amount of research on both sectors. Our literature review revealed overarching trends on the impact of corruption and transparency in these two sectors. As we searched for more specificity on the ways citizens leverage information to access services, we found the available case studies to be inextricable from their local context. For example, a seminal study conducted by Ritva Reinikka and Jakob Svensson (2005) in Uganda documented the effects of a citizen audit of a school grant received from the national government. A local newspaper completed the audit and printed its tracking of funds for local citizen consumption. Armed with this information, parents reacted by demanding that their local school leaders be held accountable for the misappropriated funds. Reinikka and Svensson

5

also claim that as a result of the community’s heightened emphasis on education, school attendance improved. The Ugandan newspaper’s audit is a clear example of citizen empowerment based on access to sector-specific information, but one that cannot be stripped of its context-specific elements. The journalists’ role in translating the information into a digestible format, the local community’s literacy rate, their access to print media, and the value that citizens place on education are all at work to make this exercise in transparency particularly impactful. This is just one example of many case studies we reviewed which alluded to potentially useful information transparency platforms or processes, but where the local nature of the results meant that the findings could not necessarily be assumed in other communities. In other words, the somewhat simplistic theory of change that underpinned our eventual indicators might not travel automatically from location to location. As we began to understand the framework within which transparency issues are discussed in both sectors, we also struggled with the knowledge that our assessment should cater to the realities of the pilot country. Our background research proved that there is no internationally accepted best practice on the ways in which healthcare and education information are transmitted to citizens. While a newspaper platform worked for citizens in Uganda in one instance, we should not assume that this is the best way to publicize information universally. The only caveats to this are the consensus standards for publishing budgetary information, which we were able to incorporate into this set of indicators. Synthesizing and Writing: Based on our desk research and input from our interviewees, Global Integrity drafted a set of indicators to assess the existence of sector-specific information we believed would be crucial to citizen empowerment in the context of participatory service delivery decision-making and social auditing; the ways this information is transmitted to citizens; citizen access to accountability systems ideally situated to defend and enforce access to that key information; and the usability of that information for discreet societal groups. To do so, we created four categories and placed the indicators within one of these different “buckets.” Due to the importance of financial resources flows in both of these sectors, we decided to dedicate a separate section of indicators to citizen access and use of education- and healthcare-specific budget information. The resultant indicator categories were:

Basic Issues around the Existence and Usability of Information in Healthcare and Education:

In this set of indictors we assessed the availability, accessibility and usability of information on healthcare and education services. (For example, we asked whether information on official fees was both user-friendly and accessible to individual citizens, civil society groups and journalists as well as whether such information was jargon-free.) We also examined the existence of information on quality of performance and whether information was standardized in a way that made it comparable across providers.

6

Redress Mechanisms that Enforce Accountability in the Health and Education contexts: We theorized that citizen empowerment through information transparency could only be achieved if there are reliable redress mechanisms that could deliver citizen complaints to service delivery agencies and ensure that complaints improved future performance (via the threat of sanctions or penalties, ideally). Localized, more intrinsic information on quality of service delivery is also spread through community and individual-based networks through “word-of-mouth.” Redress mechanisms should exist to capture these sentiments and to transfer localized knowledge to higher authorities and decision-makers.

Availability of Fiscal/Budget Information with Which to Conduct Citizen Audits of Local Schools and Clinics:

These indicators assessed the availability of basic fiscal and budgetary information that would theoretically allow local citizens (often through intermediaries such as CSOs and media) to monitor service delivery resource flows and the allocation of funds in the health and education sectors. We theorized that greater public awareness of this information would contribute to more empowered, informed citizen monitoring and participation (our fourth category of indicators).

Citizen Participation in Local Decision-Making as Influenced by Availability of Information:

These indicators were designed to assess how the availability of information impacted citizen input to local health and education policy decisions. The existence and effectiveness of formal consultative mechanisms were assessed as well as other informal mechanisms that could theoretically effectively convey citizens’ concerns to policy makers. Feedback and Refining: Our final step was to solicit feedback from relevant stakeholders within the World Bank and from members of the internationally dispersed Global Integrity field staff (more than 1,110 governance experts in more than 120 countries). We published the draft indicator set online for external review. We wanted to ensure that the indicators struck a balance for this pilot study by identifying the most relevant types of information through which citizens could theoretically demand better service while avoiding prescriptive assumptions so that local processes and manifestations could be captured. This was particularly important considering the pilot country was still yet to be determined. In their feedback, Global Integrity’s field staff shared descriptions of the conditions of their national healthcare or education systems. For example, from Japan, we learned of a government initiative to increase access to free education at the secondary level. From Georgia, we heard how internally displaced persons have little to no access to basic healthcare and education services. These anecdotal stories revealed a need for a direct analysis of the breakdown in service delivery that potentially went beyond our narrow focus on information access. One of our colleagues from Papua New Guinea (PNG) put it best when he said, “our general impression is that for PNG, at least, the indicators may be too sophisticated, when the problems are more

7

basic…” These comments confirmed our sense that while the access to sector-specific information is certainly an issue in many countries, information transparency has generally been a secondary challenge behind more fundamental issues of access and capacity. In addition to looking to experts from Global Integrity’s international network, we also welcomed feedback from the World Bank’s HDNCE. We incorporated these comments into our final draft of indicators. The full set of indicators, their scoring criteria, eventual scores, and supporting references and commentary are available on the Global Integrity wiki page (http://commons.wikispaces.com). Global Integrity Reflections on the Indicator Development Process: Through our past experience developing and fielding sub-national and sector governance and anti-corruption assessments under our Local Integrity Initiative, Global Integrity is of the mind that a truly “actionable” governance assessment is one that is locally tailored to identify the specific and relevant institutions, actors, service beneficiaries, and information types most important for driving governance outcomes in the local context. In simple terms, we shy away from employing template approaches at the sub-national and sector levels and instead prefer to develop unique methodologies each time we pilot fieldwork, borrowing useful lessons learned from other efforts when we can. While we did receive valuable input from healthcare and education experts throughout the indicator development stage, the disjuncture in timing between developing the indicators and identifying our pilot study site made it impossible for a local partner group to participate in the indicator creation process. This was a non-ideal though unavoidable situation. As a result, Global Integrity employed a more top-down approach to this pilot’s indicator development in comparison to our typical work. Despite those drawbacks, for this assessment we are cautiously optimistic that our desk research, the informal interviews we conducted, and the feedback we received from relevant stakeholders yielded a useful tool for assessing sector-specific means for citizen empowerment through access to information in education and healthcare. We have also learned that the ways in which citizens access information, the types of information most relevant to their decisions, and the processes by which they seek redress are not universal. For this reason, it is difficult to design a template of indicators to be applied to all local contexts while also expecting these indicators to yield local and “actionable” results. Despite these challenges, the study does capture some of the important ways in which information is transmitted to citizens in healthcare and education, how citizen make use of redress mechanisms to demand and enforce their rights to information, and the usability of that information for discreet societal groups. Selection of Local CSO: To identify the appropriate pilot country and local CSO to conduct this pilot study, Global Integrity actively recruited potential research teams through a formal “Call for Experts.” We received 26 applications from local CSOs around the world that expressed interest in collaborating with us.

8

After a competitive internal selection process, we decided to team up with the Macedonian CSO CRPM based on several factors, including (but not limited to): the formal qualifications of the local experts, available resources, and whether the country would make an inherently interesting case study with respect to information access in the education and health sectors. Fieldwork Methodology: Like Global Integrity’s other national-level and sub-national work, this sector-level study employed an expert assessment methodology rather than a Public Expenditure Tracking Survey (PETS)- or Citizen Scorecard-type approach. It was also not a household or beneficiary survey with a large “N” of respondents. Thus, CRPM carried out the assessment and data gathering for this pilot study by assigning scores to indicators based on original desk research and interviews with district and national health officials, healthcare beneficiaries and patients, and other healthcare experts in Macedonia. In the health assessment, the research team interviewed 26 doctors, 100 patients, and 2 health experts in the 11 municipalities that make up the capital city of Skopje. For the education assessment, the research team visited 18 schools and interviewed 88 parents, 26 students, and 14 school directors or administrators in the Skopje municipal region. Those interviews with key informants were essential to providing a rationale for indicator scores. All indicators were anchored by unique scoring criteria to promote inter-coder reliability and supported by detailed referencing and narrative commentary. We employed two types of indicators: “in law” and “in practice.” All indicators, regardless of type, were scored on the same ordinal scale of 0 to 100 with zero being the worst possible score and 100 perfect. “In law” indicators provided an objective assessment of whether certain legal codes, fundamental rights, types of information, government institutions, and regulations existed. These “de jure” indicators were scored with a simple “yes” or “no,” with “yes” receiving a 100 score and “no” receiving a zero. “In practice” indicators addressed the “de facto” issues such as implementation, effectiveness, enforcement, and citizen access to those same “de jure” mechanisms and institutions. As scoring those indicators usually requires a more nuanced assessment, these “in practice” indicators were scored along an ordinal scale of zero to 100 with possible scores offered at 0, 25, 50, 75, and 100. Peer Review Process: Two peer reviewers were contracted to review the draft data and provide standardized feedback to help make corrections and adjustments to the final data when necessary. This quality control mechanism helped to ensure that our data were as accurate and balanced as possible. These peer reviewers were vetted for their independence and expertise in Macedonia. The reviewers who were selected had served in the same capacity for previous national-level Global Integrity Report: Macedonia assessments. As with all Global Integrity assessments (national-, sub-national-, sector- level), the peer reviewers were asked to attend to the following:

• Was the particular indicator or sub-indicator scored by the lead researcher factually accurate?

• Were there any significant events or developments that were not addressed?

9

• Did the indicator or sub-indicator offer a fair and balanced view of the sector and information transparency environment?

• Was the scoring consistent with the entire set or sub-set of integrity indicators? • Was the scoring controversial or widely accepted? Was controversial scoring sufficiently

sourced? • Were the sources used reliable and reputable?

The peer review process for the data scorecard did not assign direct attribution to peer review comments. This ensured that peer reviewers were unrestrained in their commentary. Peer review comments on the country’s data scorecard were published alongside the final scorecard and played an important role in final scoring adjustments prior to publication. With regard to peer review feedback on the scorecard data, peer reviewers were offered one of two standardized choices in responding to a given indicator or sub-indicator, using the above guidance to evaluate each data point:

• “Peer Review Score Change (YES or NO).” Peer reviewers were asked whether they agreed with the score assigned by the lead researcher.

• “Peer Review Comments.” Peer reviewers were encouraged to provide optional comments in order to add contextual depth to the score. If they expressed disagreement, they were asked to explain and defend their criticism of the score and suggest an appropriate alternative score or reference.

The peer reviewers for the Macedonia pilot study did not register any significant disagreement or reservations with the data. On the contrary, the reviewers corroborated most of the findings with only a handful of suggestions focused on minor score changes (e.g., raising the score to 50 from 25) based on the reviewers’ judgment and their interpretation of the research team’s comments. In other instances, the reviewers also included additional comments to support the score for a particular indicator. For example, in the health assessment indicator asking whether, “in practice, mechanisms or processes exist through which citizens can access information on the rules for hiring, firing and rewarding doctors” (which earned a 50 score), a reviewer noted: “while it is easily accessible, information on the rules for hiring, firing and rewarding doctors does not include politically motivated hiring and firing practices.” Global Integrity staff also performed their own review of and corrections to the data before and after the peer review process, often requesting that the local research team bolster their interviews and references in order to ensure accurate scoring. The baseline level of quality we sought to ensure in all indicators was for a non-expert in Macedonian healthcare and education to be able to publicly defend each and every indicator score based on the comments and references provided. CRPM Reflections on the Indicator Development and Data Gathering Process: The main conclusion drawn from this pilot exercise by the CRPM research team was how well the indicators as a whole corresponded to local level realities. Indeed, they found that most of the indicators were relevant (to varying degrees) to the Macedonia context, which suggests that

10

several of the questions in this initial study could potentially feed into future country pilots despite the more top-down approach to indicator design that we employed. For instance, based on our desk research and informal interviews as well as our previous experience working on other Local Integrity projects, Global Integrity decided it would be important to include a few indicators on the role of informal mechanisms and processes to information transparency and dissemination in education and health service delivery (e.g., “in practice, citizens make use of informal or alternative processes or routes to lodge complaints on wait times in clinics [for example, seeking advice from religious or other civic leadership]”). Based on CRPM’s responses to these indicators, it was evident that non-formal arrangements were indeed important to service delivery vis-à-vis information sharing in the health and education sectors (see the sub-section “General Findings” on page 12 for further discussion). This cluster of indicators was one example, among others, that the tool we generated was robust enough to resonate well with Macedonia. Another example was the set of indicators focusing on citizen audits (e.g., “in practice, audits of local health clinic budgets are conducted by citizens”). Once again, based on our review of the existing literature of “best practices” as well as informal interviews, Global Integrity came to the conclusion that it would be worth asking questions about these demand-side mechanisms because they could serve as a useful cross-country benchmarking tool to catalogue which countries do and do not benefit from such participatory mechanisms. Although the CRPM researchers found such questions to be irrelevant since these groups currently do not exist in Macedonia, they nevertheless recommended that they be retained because they could inform future citizen advocacy initiatives. Conducting country comparisons of citizen- or community-driven audit and accountability initiatives could bolster these advocacy efforts. One specific area that the research team suggested could be fleshed out were indicators related to the health and education systems’ inclusiveness toward vulnerable groups, minorities, and women. This suggests that future indicators should be explicitly disaggregated by citizens’ ethnic, religious, gender, and class backgrounds (see the sub-section “What’s Missing from the Data” on page 13 for further discussion). The CRPM researchers’ advice on completing this study in other countries focused less on the indicators themselves than on their structure or organization. They suggested creating a separate section for indicators focused on issues that do not vary greatly from one local governance unit to the next (e.g., municipalities), or where local level disaggregated data are not available. This may also facilitate cross-country comparisons. Another section could concentrate on indicators with specific local level (e.g., municipal-relevant) data, which could enable intra-country comparisons. III. PRELIMINARY FINDINGS FOR MACEDONIA We begin our discussion of the country findings for Macedonia with some general observations. It is followed by a consideration of some lacuna in the indicator design and data of this pilot project.

11

General Findings: Five primary themes emerge from the research. The first is the yawning gap between information access laws as they relate to education and healthcare service delivery and their implementation and enforcement in Macedonia. To be sure, a substantive portion of the relevant legal framework is in place, but significant work remains to be done in order to ensure that it is effectively realized. One likely factor that contributes to the implementation gap is the lack of coordination among different policy making institutions, which has created a system lacking in transparency that weakens enforcement of the right to health care. The disjuncture between formal rules and how they operate in practice can also be attributed, in part, to the unstable processes accompanying Macedonia’s ongoing accession to the European Union (EU). As one of our peer reviewers notes, while many legal reforms in Macedonia have been conducted in line with requirements for EU advancement, they are what he calls “Catch 22” laws: that is, the laws themselves contain everything that outside partners have asked for, but the “de facto” reality reveals these laws to be inadequately enforced or implemented. By way of example, he points out that if information access in a particular service delivery sector in Macedonia, such as healthcare or education, is not governed by a law or regulation specific to education or health, the solution is to fall back on another law that is not directly related to education and health, such as the Law on Free Access to Information or the Law of the Ombudsman. What was clear from the data gathered during this pilot project is that defaulting to national-level information access mechanisms rarely yielded positive outcomes for beneficiaries. In lay terms, national-level access to information/right to information laws may not be a panacea for ensuring information access at the sector level even if a national law theoretically provides for such access. Such national laws and mechanisms are simply too cumbersome and too removed from most beneficiaries’ daily habits as to make them useful. For instance, although the public has the legal right in Macedonia to access school budgets, this right is regulated by a budget law rather than a law on education. As a consequence, while budgets are made available to the public, they do not provide full details including an itemized list of budget allocations. We observed a similar disconnect with regard to complaint mechanisms. It is one thing to claim that there is a mechanism for citizens to voice their complaints about a particular problem in education or healthcare, but it is completely different to say that such a mechanism is present because there is a general mechanism for public complaints about government performance embodied in the Law on Ombudsman. To quote our peer reviewer: “the possibility to have access to that information under the law on free access to information is only theoretical. The law is ‘dead letter on paper’ – the law is not respected or enforced by the very people who should provide the information requested. Every attempt to force them to act differently [has been] unsuccessful. So if you use the Law on Free Access to say that the information is available, it is only mockery.” The second significant theme is the relative absence of demand-side governance mechanisms that facilitate citizen participation such as citizen audit and monitoring groups in both the health and education sectors. Citizen engagement and local empowerment through participation in key decision-making processes as well as auditing and monitoring exercises are generally understood (by transnational agencies) to be a core part of democratic governance. In the case of Macedonia,

12

however, civil society groups that oversee health clinics and schools are virtually absent, although an exception was noted by CRPM in the case of the Organization of Women from Sveti Nikole who successfully monitored the implementation of women’s health preventative programs. Despite the lack of robust citizen audit and monitoring groups, however, a word of caution is in order lest our comments imply a judgmental evaluation. The emergence of citizen groups in the form of community or civil society organizations must be put into a broader contextual framework. In the field of healthcare, for instance, the policy decision-making process in Macedonia is still heavily centralized, which, in turn, has made it difficult for grassroots civil society groups to thrive. Arguably, these mechanisms will be unlikely to gain much traction until the state offers greater support to civil society and the healthcare system devolves in a vein similar to what is happening in the education sector (although here, too, citizen/community groups are weak to non-existent).1 An even simpler reality is worth mentioning: Skopje, Macedonia is not Rajathstan, India, arguably the epicenter of the world’s most successful citizen monitoring efforts on information transparency. The success of other citizen auditing and social monitoring mechanisms in other countries should not be assumed to travel well given vastly different social, economic, and cultural frameworks. The third organizing theme is that while few citizen/community-based monitoring groups in the health and education sectors currently exist at the sub-national level in Macedonia, official government audits and monitoring of health and education are conducted fairly regularly. In education, the State Education Inspectorate evaluates schools on a revolving schedule to account for the large volume of schools to be reviewed. The data gathered here suggests that while all criteria may not be strictly followed in all government audits, monitors are fastidious about spending the required amount of time visiting each school. The assessment found that the results of official school reviews are acted upon and lead to consequences for the schools. This differs from citizen monitoring of education, which does not factor into school-based decision-making in Macedonia, partially because citizen monitoring is typically conducted at the national and not the sub-national or school level. One interesting reason why citizen auditing typically occurs at more elevated levels of government is due to the type of school-specific financial information made available to citizens. The school budget is not required to be published on its own. Instead, budget information is provided by schools to the municipal governments to be included in a more general summary of local school spending as part of the published municipal budget. Financial transparency is also lacking regarding government audit reports, which are not regularly published. In health, government monitoring responsibilities over health clinics are fragmented and distributed among three entities: the Health Insurance Fund (HIF), the Public Health Institute, and the Health Inspectorate. Audit reports are made public and are available on the HIF’s website, and there have been documented cases where health clinics have been closed because of unhygienic conditions as a result of such audits.

                                                                                                               1  There are a few signs that recent initiatives supported by the World Bank to reform the health sector have increased the involvement of civil society, including the representation of civil society groups like the Pensioners’ Association on the Board of the Health Insurance Fund.  

13

The fourth major theme of the research is the role of informal mechanisms and processes in information sharing and dissemination as they relate to service delivery (and, we suspect, in other facets of everyday life in Macedonia more generally). Given the range of institutional as well as societal/political economic factors that have adversely affected the provision of public services in the country, it is noteworthy that citizens have turned to informal institutions and relations when formal governance arrangements have failed to respond adequately to their needs. For instance, rather than filing formal complaints about the conditions of local healthcare facilities, “more often than not, people tend to complain verbally to family members and friends and do not lodge formal complaints.” The patients interviewed by the researchers support and expand this view: “there is a prevailing public opinion that if you complain you continue to be tied up in administrative procedures because the institutions are slow and inefficient; therefore, people turn to their ‘connections’: people who know people that can push a resolution of a problem rather than using the formal redress mechanism.” Kinship relations and social networks are thus crucial factors in explaining particular development outcomes. In education (and perhaps in other sectors as well), the media in Macedonia “has, by default, been used as the informal or alternative” method to lodge complaints about school-related issues such as school meals. Indeed, it is potentially significant that the media has assumed its role as the default redress mechanism for many citizens. The media was an important propaganda tool in the civil war that engulfed 1990s Yugoslavia. After the conflict ended, international donor agencies like USAID focused heavily on the media since they viewed the promotion of media professionalization and pluralization as part of a broader post-conflict resolution program to help with community mobilization, development, and participation (Broughton-Micova 2006). It is worth thinking through how the media has enhanced and informed an engaged citizenry (or not) in Macedonia. Although institution-building has been the focus of most development, democracy, and good governance actors, the research suggests that this conventional wisdom should be questioned in light of evidence here and elsewhere of the constitutive role that informal institutions and social relations play in information access vis-à-vis service delivery in key HD sectors. This has implications for policy makers who may wish to consider how informal local governance can complement and strengthen rule-based institutions (Institute of Development Studies 2010). The fifth cross-cutting theme is how information is transmitted, which significantly determines whether citizens can access information that is publicly released. Where information is available, such as citizen complaints of health facilities or a variety of school-based issues, they are typically done so online. Online access can be a barrier for poorer and rural-based citizens, many of whom are included in the forty-nine percent of the country lacking internet access. The unit of analysis for this study was the capital city of Skopje, so rural access issues were not extensively considered. The references to national-level mechanisms to access information, such as the national Ombudsman’s office, may also be more available to citizens of the capital city than elsewhere. What’s Missing from the Data:

14

There are two significant gaps identified in our pilot assessment. The first is the relatively thin data on local health and education service delivery issues disaggregated by citizens’ ethnic, gender, religious, and class backgrounds. It is worth bearing in mind that the reform of public service provisions in key HD sectors overlaps with the Ohrid Framework Agreement (OFA), which Macedonia adopted in 2001 to resolve ethnic conflicts through a decentralization process that distributes power between ethnic Macedonians, ethnic Albanians, and other minorities at the local level (Lessenski et al. 2006). Moreover, municipalities with 20% or more minorities (mainly, Albanian) are bilingual by law in accordance with the section of Macedonia’s Peace Accord pertaining to the use of minority languages. Therefore, citizens’ religious/national/ethnic/gender identities are likely to structure the kinds of services they receive (or are excluded) from their local governments and municipalities since the politics of ethnicity, religion, and gender have historically been central to this country and region (Brown 2006, 2003; Dimova 2007; Neofotistos 2004). Despite the legal enshrinement of bilingualism, our assessment, identifies a few areas of information transparency where ethnic cleavages have been of concern in practice (e.g., patient rights are publicly available in the Macedonian language, but not in the Albanian language; the Roma and homeless populations are not adequately informed of the types of health services available to them; Roma school children are forced to study the majority language of the districts they live in). Given the importance of ethnicity in Macedonian social life, more data on the practical implementation and enforcement of bilingual laws should ideally be available for a country like Macedonia. The second interrelated issue that we feel is underplayed in our data is regional differences within the country, particularly between rural and urban areas of Macedonia. These variations very likely shape how citizens can or cannot access information on healthcare and education service delivery, but these data were harder to obtain in this exploratory study since it was limited to the Skopje municipality. As we noted earlier, for example, the CRPM research team demonstrates that several governmental agencies release information on their websites in a country where internet penetration is higher in urban areas than rural ones. Furthermore, religious, class, and ethnic differences in urbanized and more underdeveloped rural regions also structure the kinds of services and information access citizens receive from their local governments. Expanding the project scope to encompass indicators sensitive to the urban/rural divide can help disaggregate the monolithic category of “citizen.” In so doing, more specific information on citizens’ ethnic, gender, religious, and class identities can inform how certain groups of citizens actually experience information transparency in health and education service delivery. IV. FUTURE DIRECTIONS In this final section, we close with some thoughts on the efficacy and limits of these sector-specific indicators and look ahead to potential future steps. What These Indicators Can Tell Us: We propose (at least) three ways that “going local” can be useful. The first is that it enables us to assess and map the information transparency situation in a local context in health and education. A focus on this issue at the national-level or through cross-country comparisons alone obscures

15

what is really happening around citizen access to information in these key HD sectors. Indeed, the key issues being played out are happening at the local level, and we suggest that it is through this optic that we can make better, evidence-based policy assessments and choices that are contextually appropriate for a country. Consider, again, our preliminary finding that citizens in (the municipalities of) Macedonia rarely utilize information-related redress mechanisms and frequently turn instead to informal social networks. Based on this finding, we might do well to revisit the assumption that mechanisms for seeking information and redress are useful to “measure,” or at least assess based on conventional measures of institutional performance without a concomitant examination of informal processes.

Second, these indicators can help pinpoint legal and implementation gaps that could be addressed by policy reform and improved enforcement/implementation. The Janus-faced character of information access in health and education is particularly pronounced in the case of Macedonia where, as one of our peer reviewers notes, several laws that have been designed to meet EU accession requirements are unevenly enforced. For instance, although there is a Law on Freedom of Access to Information, not all institutions release the required information regularly. Information on healthcare budget allocations is made available only upon request rather than being proactively published. It is this kind of disaggregated, concrete, and targeted data that has the potential to assist policy makers in closing implementation gaps. Third, although these are “quantitative” indicators, they are accompanied by narrative commentary to draw out slightly thicker contexts to the numerical scores. These comments, which are supported by references to interviews or published materials, provide nuance to issues in service delivery in health and education that are inherently complex and often deceptively simple on the surface. For instance, while there is a formal redress mechanism in Macedonia for citizens to file complaints (the Ombudsman’s office), on closer inspection, there is no specific agency that deals with education- or health-specific issues, which makes the Ombudsman potentially less effective in dealing with sector-related complaints. Furthermore, the qualitative data reveal that when complaints are filed, citizens expressed most concern about fees and payments/quality of health services (health) and school meals and curriculum (education). We firmly believe that data drilled down to specific details provide actionable information with which to make more informed policy decisions. What These Indicators Cannot Tell Us: One of the challenges that any producer and user of indicators face is how to interpret the results. Indicators are often asked to perform more interpretive work than they are capable of with relatively little acknowledgement of their limits. This challenge holds true with our assessment as we consider what effect, if any, information access has on service delivery and “citizen empowerment.” Based on the partial evidence from this pilot study, we suggest that isolating the effect of “access to information” on better governance and citizen empowerment in health and education service delivery remains indeterminate. The well-worn phrase “knowledge is power” cannot be easily applied to Macedonia; that simplistic theory of change overly simplifies a more complicated situation in which citizens do not always seek information or information redress through official, institutionalized channels. A deeper “power” analysis would be required by, at the very least, combining an examination of information access with the four other pillars

16

identified by the authors of “Making Services Work” (i.e., financing systems, human resources, critical inputs, and provider entry). We can then better comprehend how social transformation happens through an examination of congeries of financial, political, economic, and socio-cultural factors that impinge upon and shape the governance of service delivery. In doing so, it might be possible to identify the relative impact information access has on service delivery governance and citizen empowerment in relation to other components or pillars in the delivery chain and beyond. This approach therefore militates against the assumption that isolating, examining, and intervening in one key area, such as information access, via indicators alone can lead to quick and improved outcomes in the governance of service delivery. Rather, it suggests that social change is a complex, relational, and multi-layered process that occurs slowly, unevenly, and incrementally over time (Lampland 1991), a conceptual framework that is perhaps best condensed in contextually “thick” case studies to which we now turn for further elaboration. Potential Next Steps: Based on the discussion above, we suggest that the next phase of research should focus not only on redeploying these indicators (which we speak to below) but also on potentially deploying complementary “thick” case study analyses to try and unpack why (or why not) information transparency matters in driving improved/deteriorated service delivery in key HD sectors such as education and health.

In future country pilots, we recommend the following: a) Deploying a core set of indicators that we suspect would be universally applicable based on our experience with sub-national indicators more generally and this Macedonia sector-specific pilot in particular. This set could map:

• The access to information legal framework and types of information available (e.g. Are the local health clinic budgets made available to citizens? Do basic patients’ rights exist and are they publicly disseminated? Is information on school curricula made available to citizens? Does a code of conduct exist for teachers, covering issues such as prejudice and favoritism in the classroom, school attendance, communication with parents/guardians and fee collection? Is the code of conduct available to citizens?)

• Informal redress and accountability mechanisms (e.g. parent leaders, village

heads, religious groups, civic gatherings) • Citizen auditing and other forms of citizen/community participation that rely on

information access (e.g. indicators exploring the existence and effectiveness of PTAs and PTA meetings, state inspections, hotlines, ICT-based redress, service provider HR offices, government offices, and school monitoring committees); and

• Budget information indicators (e.g. Are there formal mechanisms in place

documenting and tracking resource flows from both state and non-state sources? Are

17

resource flows documented through the entire source-to-receiver chain? Are data about resource flows available to the public (CSOs, media, and/or individuals?)

b) Employing a more bottom-up process, through explicit co-designing of indicators with local partners, to complement that core set of indicators with locally-specific indicators to assess:

• Formal complaints mechanisms related to information access (e.g. these can vary widely from local context to local context to potentially involve/include ombudsmen, direct line supervisors, government auditing bodies, and/or locally elected councils or officials);

• Disaggregation of information access issues by ethnicity/gender/income.

We feel that piloting a core set of universal indicators with a set of satellite (i.e., local) indicators in one or two other countries could help to further refine this approach into a tool that could be usefully replicated in a number of contexts for benchmarking and policymaking purposes. In certain cases, short but useful qualitative case studies could also be used to begin unpacking why information access in certain countries is, or is not, contributing to beneficiary empowerment and/or improved service delivery. Such a qualitative approach need not be expensive or overly time-consuming and could model itself (in a more concise manner) along the lines of the political economy case study approaches deployed by the UK Department for International Development (DFID) and the Swedish International Development Cooperation Agency (SIDA). Subject to the availability of suitable local partners, future pilot countries could include:

• Argentina • Egypt • Ghana • India • Kenya • Papua New Guinea • Peru • Philippines • Ukraine

Lastly, attention should be paid in any future pilots to issues of heterogeneity. If users of this tool are concerned with heterogeneity within a country context, it is possible (provided sufficient resources and time are available) to field this tool across multiple “targets” (cities, provinces, regions etc) to explore whether significantly differing data appear on similar issues across different targets. Within a particular target, the heterogeneity issues are slightly different. Our sense is that by employing the more disaggregated approach across ethnicity, gender and income (described above) to some of the indicators will help to account for heterogeneity within a particular target. Although we would not expect this approach to yield scientific or empirical precision, we do expect it to help identify targets where heterogeneity does present a major methodological challenge to this expert assessment approach; such a complex situation could call

18

for supplementary “large N” household surveys to reinforce (or debunk) data gathered via the expert assessment.

19

References: Broughton-Micova, Sally. 2006. “Confidence-Building, Market Forces and the Public Good: Challenges of Media Intervention in Postconflict Macedonia.” In Transacting Transition: The Micropolitics of Democracy Assistance in the Former Yugoslavia, ed. Keith Brown, 125-141. Bloomfield, CT: Kumarian Press. Brown, Keith, ed. 2006. Transacting Transition: The Micropolitics of Democracy Assistance in the Former Yugoslavia. Bloomfield, CT: Kumarian Press. Brown, Keith. 2003. The Past in Question: Modern Macedonia and the Uncertainties of Nation. Princeton, NJ: Princeton University Press. Dimova, Rozita. 2007. “From Past Necessity to Contemporary Friction: Migration, Class and Ethnicity in Macedonia.” Max Planck Institute for Social Anthropology Working Papers, Working Paper No. 94. Fiszbein, Ariel, Dena Rinegold, and Halsey Rogers. 2009. “Making Services Work: Indicators, Assessments, and Benchmarking of the Quality and Governance of Public Service Delivery in the Human Development Sectors.” World Bank paper. Institute of Development Studies (IDS). 2010. An Upside Down View of Governance. IDS Centre for the Future State. Lampland, Martha. 1991. “Pigs, Party Secretaries, and Private Lives in Hungary.” American Ethnologist 18 (3): 459-479. Lessenski, Marin, Antonina Habova, and Vladimir Shopov. 2006. “The Process of Decentralization in Macedonia: Prospects for Ethnic Conflict Mitigation, Enhanced Representation, Institutional Efficiency and Accountability.” Institute for Regional and International Studies, Institute for Democracy “Societas Civilis” Skopje, and Freedom House- Europe. Neofotistos, Vasiliki. 2004. “Beyond Stereotypes: Violence and the Porousness of Ethnic Boundaries in the Republic of Macedonia.” History and Anthropology 15 (1): 47-67. Reinikka, Ritva and Jakob Svensson. 2005. “Fighting Corruption to Improve Schooling: Evidence from a Newspaper Campaign in Uganda.” Journal of the European Economic Association 3 (2-3): 259-267.

20

Appendix Specific Findings of Interest from the Indicators

Health: General Findings: In contrast to education, Macedonia’s healthcare system is still quite centralized. Although many laws governing citizen access to information on health exist de jure, their implementation and enforcement are inconsistent. This large implementation gap is particularly glaring with respect to citizen redress mechanisms and the availability of public information. In Macedonia, citizens turn to the national Ombudsman office to file complaints since there is no health-specific redress mechanism and the Ombudsman is considered the general complaints mechanism. Complaints received by the Ombudsman reveal that, in general, the two biggest concerns voiced by citizens with respect to information access on local healthcare service delivery are: 1) fees and prices of health services, and 2) the quality of health services, including the state of cleanliness of health facilities, the non-reimbursement of funds by the HIF, and the unreasonable timing of reimbursements. However, because the Ombudsman is generally perceived to be bureaucratic and ineffective, citizens often turn instead to the media, social networks, and consumer organizations/NGOs to voice their complaints and seek redress. Another significant gap in the healthcare sector can be found in the public release of information. While there is a general Law on Freedom of Access to Public Information, not all institutional bodies publicize their information on a regular basis. For instance, information on issues such as budget allocation is only available upon request by citizens through the Law on Freedom on Access to Public Information. Furthermore, several patients interviewed by the CRPM researchers believe they are not adequately informed about their local health clinic’s budget, even though clinics are legally mandated to do so in their contract with the HIF, which does publish the clinics’ financial reports on their website. Other issues, however, are publicized regularly. This includes prices and fee structures for health services. In addition, the Ombudsman keeps records of all complaints lodged in their office and publishes them in an annual report that is available on their website. As Macedonia tries to harmonize its healthcare policies with EU accession requirements, we should expect that the information access laws will continue to be inconsistently applied. Specific Findings: Bucket 1: Basic Issues Around the Existence and Usability of Information in Healthcare: - Many patients find what information that is available to be laden with jargon and non-user friendly. Where information is lacking, they attribute it to the highly politicized environment of the healthcare system, such as the hiring/firing of doctors and other healthcare workers. - The lack of information affects minority and disadvantaged groups in Macedonia disproportionately, in particular the homeless and Roma population. - Primary healthcare is privatized and provided through a network of service providers. These providers receive cash transfers from the HIF for the services they provide. The contracts with

21

the primary health providers, which are all privately managed, include the budget ceilings and are publicly available on HIF’s website. Bucket 2: Redress Mechanisms that Enforce Accountability in Health contexts: - Formal procedures to lodge complaints are very time-consuming. - The Ombudsman is the general complaint mechanism. There is no health-specific agency to deal with citizen complaints. - Informal groups, CSOs, and the media are the alternative sources for information access and dissemination. - Official government monitoring is fragmented: it is divided among three institutions (HIF, Public Health Institute, and the Health Inspectorate), so monitoring is not standardized. - Citizen complaints that are received through official channels are mainly related to fees/prices as well as cleanliness of health facilities. Bucket 3: Availability of Fiscal/Budget Information with Which to Conduct Citizen Audits of Local Clinics: - Specific reports on this issue only do not exist, but information is provided in the budget documents (when the resources are planned) and in the final accounts of the clinics (when the resources are executed). The budget documents and the final accounts are public documents but are not disseminated widely (on official bulletins and etc.). Citizens may ask for this information using the procedures and the forms for free access to public information (regulated by the Law on Freedom of Information). The two documents are considered as not very user-friendly as they show the data in tables and use technical language. - There are no citizen audits. Bucket 4: Citizen Participation in Local Decision-Making as Influenced by Availability of Information: - Health policy is not decentralized and therefore community forums are not directed towards stirring debates on health. - There are currently no examples of information campaigns for the rights of citizens to associate and participate in patient advocacy groups. Education: General Findings: As in health, much of the basic information transparency legal framework is in place to govern access to information in education, but the enforcement and implementation present a more

22

mixed picture. There are no education-based legal rights to information; however information specific to education can, in theory, be accessed by citizens through the Freedom of Information Act. The assessment notes that in the education sector in Macedonia, two new types of information are being developed. In 2007, schools were urged by the national government to create professional codes of conduct for school administrators and teachers. In schools where these codes do exist, they are generally published online and can be requested under the citizen right to information law. Also under development are national education standards which will provide parents with a clearer set of expectations for school performance and allow for comparability between schools. These standards have begun to be adopted by the State Education Inspectorate for government monitoring purposes. The assessment acknowledges citizen concern in two particular areas related to education: curriculum and school meals. Citizen involvement in curriculum development is low and parents are dissatisfied with the level of details relayed to them on how curriculum goals are expected to be instituted in the classroom. Citizen pushback is also acknowledged in relation to institutionalized language of instruction at schools. Local communities used to dictate their school’s language of primary instruction based on the ethnic majority. However a recent law mandated that starting in first grade, all students be instructed in Macedonian. A constitutional court case ensued from this decision and eventually overturned the law. In Macedonia, no national school meals program exists but at the school-level, parents can collectively choose to pay for school meals. Complaints about the quality of the meals are high and there is little information provided on meal availability or on their nutritional value. Interestingly, the decision-making processes for both curriculum and school meals are partially decentralized to the school level. The fact that this assessment highlights citizen dissatisfaction surrounding these services does not necessarily signal dissatisfaction with more local participation processes, but may instead indicate that in education in Macedonia, citizens are generally not aware of processes for participation on the national-level. A legal framework to establish education-specific citizen redress mechanisms and information availability procedures, as with healthcare, does not exist in Macedonia. To access education-related information, citizens can make use of their general right to information at either the national, municipal or school-level—depending on the information requested. To lodge an education (and, for that matter, health) -related complaint, the most applicable mechanism identified in this assessment is the national Ombudsman’s office. However, citizens rarely turn to this office for education-related matters as can be seen by the low number of education-related complaints in the past year: twenty-six. This is partially due to the inefficiency of the Ombudsman’s information campaign publicizing the office’s involvement in education and children’s rights. On the school-level, parent council meetings are well-attended. These organized school meetings provide a forum for citizens to both receive information and to air their complaints, especially around school curriculum. The media can also be a forum for citizen complaints. Recent articles have been published on the poor quality of school meals provided for students. CRPM’s survey on parents’ perceptions on the accessibility and user friendliness of information related to education-focused issues (such as school meals, curriculum, standards and quality of

23

service and professional codes of conduct) showed the overall perception of information clarity to be low. In sum, much of the basic information transparency legal framework is in place to govern access to information in education, but the enforcement and implementation present a more mixed picture. Furthermore, despite the emphasis on decentralization in the education sector in Macedonia, the results show that, in general, in order to seek redress or to access information, citizens must use national-level mechanisms. Municipal, and city mechanisms are rarely mentioned in the results. School-based requests can be made on an ad hoc basis, secondary basis should information not be delivered to the citizen by the national information office. Specific Findings: Bucket 1: Basic Issues around the Existence and Usability of Information in Education: - Although information requests can be made at the national-level for education and school-specific documentation, these mechanisms are not generally used. - There are no systematic national codes of conduct for teachers and school administrators; yet, individual schools can adopt their own codes of conduct. - Citizens can go to school directly to request information or to a municipal or national access to information process. In cases where the school has not made the information available to the municipal or national level, citizens may be told to direct their requests to the schools. The schools should then honor those requests based on FOIA. - Teacher attendance records are not published as part of school or municipal education reporting. They are, however, included in the labor statistics. This disjuncture makes it hard for citizens to access them. - A perceptions survey undertaken by CRPM found that the majority of parents do not consider education-focused information to be user-friendly or accessible. Bucket 2: Redress Mechanisms that Enforce Accountability in the Education context: - The national ombudsman has an information campaign on children's rights and education. This is theoretically a major redress mechanism available to citizens and it is cited throughout the assessment. Although statistics show it is rarely used for education-related complaints. - Non-official or alternative complaint mechanisms: The media are sometimes sought out to publicize service delivery complaints. City-based education NGOs get a brief mention but do not seem to play much of a watchdog role. - There are currently no citizen audits or monitoring bodies. However, the government State Education Inspectorate (SEI) does monitor the quality of education and enforcement of education laws.

24

- Of all the education-related issues covered in this survey, complaints of overcrowding in classrooms is identified as least likely to be reported to the national Ombudsman’s office. - Curriculum-based complaints are most likely to be brought before the local school board, although, technically, they can be addressed by the national Ombudsman. - School meal decisions are made at the local level, but the results show that the complaint mechanism associated with this issue to be the national Ombudsman, not a more decentralized institution or outlet. - School-based monitoring is highly emphasized and completed by the State Education Inspectorate. Specific education standards are being developed and tied into specific consequences based on monitoring results. Bucket 3: Availability of Fiscal/Budget Information with Which to Conduct Citizen Audits of Local Schools: - Budget Documentation and Transparency: Schools are required to report budget information to their municipalities but not required to make a separate report. School budget information is published as part of the municipal budget. Some schools choose to publish their specific budgets online, but this is not the norm. Specific information on the transfer of resource system is not made public, but can be pieced together by comparing municipal budgets and municipal final account documents. (See also Bucket 1for related indicators.) - Citizen auditing is not practiced and there are very few local level education NGOs to assist in this monitoring. - Internal auditing is said to occur on a regular basis although no records exist to verify the implementation of consequences or changes based on the results of these internal audits. Bucket 4: Citizen Participation in Local Decision-Making as Influenced by Availability of Information: - Parent Council meetings are well-attended and two highly visible members of this group are appointed to represent parents on the School Board. However, reports are not available if the councils relay concerns to the school board or how and if the board addresses these concerns. - Information on school curricula is significantly shaped by regions: it is easier to access such information in urban areas where internet penetration is higher than found in rural areas. - There are two areas where citizen concern is acknowledged: curriculum and school meals. Indicators in Bucket 4 note the lack of citizen involvement in curriculum development. The low level of details on how curriculum goals are expected to be instituted in the classroom is also noted as a concern. In Macedonia, no national school meals program exists but at some schools parents can choose to pay for school meals. Complaints about the quality of the meals are high

25

and there is little information on meal availability or on their nutritional value. (See also Bucket 1.) - In some cases, where students who live in one municipality need to be bussed to another for school access, the two municipalities are expected to share the costs of this transportation. - There are new curriculum requirements mandating that children of all ethnicities start learning the Macedonian language in 1st grade. - Roma students are forced to study in the language that the majority of the children in the area favor: Turkish, Albanian or Macedonian.