Knowledge systems: Evidence to policy concepts in practice

In today’s world, the issues governments grapple with have become increasingly complex. Developing and articulating effective, multifaceted solutions require production and application of knowledge to support evidence-informed policies for socioeconomic development especially given the overwhelming deluge of data and information produced globally. However, connecting researchers to policy makers has proven to be a challenge. Researchers may not frame their inquiry in ways that meet policy makers’ expectations or needs, and policy makers may not communicate clearly to researchers what they want or need. This mismatch has led some to describe their relationship as “an unrequited love affair.”¹

What makes for a happier match, and how to create or facilitate a mutually beneficial process that leads to the effective incorporation of knowledge into policy? Answers to these questions have been posed from a variety of viewpoints; among the more recent ones is what can be termed “knowledge systems.”

This policy brief reviews the evolution of knowledge-to-policy studies and the emergence of systems perspectives. Theoretical and analytic assessments of knowledge systems have identified their elements and interconnections, yet less well understood or documented has been how to grow and reinforce knowledge systems in settings where they are weak and underdeveloped. We offer a knowledge systems model that encapsulates current thinking and present an example of an effort to strengthen a knowledge system, drawn from a project managed by RTI in Indonesia. We conclude with some recommendations for knowledge systems strengthening.

A knowledge system, sometimes referred to as a knowledge sector, is a holistic conceptualization that specifies, for a given country, a set of knowledge institutions and actors and delineates the interconnections among them.^2,3 The main components of a national knowledge system include the following, which are noted again throughout this brief:

• The supply of evidence or researchers (knowledge producers) consisting of universities, research centers, and think tanks^3,4,** Evidence can also be supplied through local knowledge captured through traditional systems and cultural practices. Local knowledge can also be used to influence policy.

• The demand for evidence by policy makers (knowledge users) consisting of government ministries or line agencies, and parliamentarians

• The debate about evidence through public discourse around policy issues consisting of civil society organizations, the private sector, and the media (knowledge intermediaries)

• The funding and regulations around generation of evidence consisting of regulatory authorities and public and private funding bodies (knowledge enablers)

The interactions and linkages among these components and the quality of their exchanges influence the extent to which knowledge can effectively inform policy making. These dynamic interchanges over time will have an impact on the functioning of the knowledge system. Positive interchanges can increase the likelihood of sustained uptake of research by policy makers; negative interchanges can lead to the sad “love affair” referred to above. This model is illustrated in Figure 1.

Although seemingly simple in its presentation, the knowledge system model presented in the figure is actually a model of contestation in which the different actors in the knowledge system debate policy informed by evidence, taking into consideration values, traditions, and political calculations.⁵ By supporting the interaction between actors, contestation of evidence is increased, leading to more robust thinking and decision making.

Figure 1

Knowledge system model

Source: Developed by authors, building on UK Department for International Development (DFID)² and Australia-Indonesia Partnership for Pro-Poor Policy.³

Concern with influencing policy makers through research has a long history. Numerous studies have examined how and why policy makers use research and have explored the dynamics of the application of policy analytic tools to socioeconomic problems facing politicians.^6,7 This investigative stream has continued up to the present in the form of arguments for, and analysis of, evidence-based policymaking, bringing together researchers, practitioners, and policy makers (see, for example, the Africa Evidence Network, https://www.africaevidencenetwork.org/).

Building on the findings of early research utilization studies, later investigations sought to identify the factors that affect whether policy makers are likely to refer to research when making and implementing policy choices. Various knowledge-to-policy studies converged in identifying the following factors as central to research uptake: (1) political context (participatory structures and processes, power distributions, openness to change); (2) actors’ values, beliefs, and interests (willingness to consider policy alternatives, ideological orientations); (3) types of knowledge used in policy discussions (expert vs. citizen knowledge, research-based vs. practical); and (4) structures and processes shaping knowledge-policy interactions (linkage and intermediary functions).⁸ These studies also identified factors associated with researchers that affect research uptake, such as incentives for researchers to pursue studies leading to academic and professional advancement rather than practical application, concerns with methodological debates that overshadow attention to policy relevance and practice, and the weak ability of researchers to communicate effectively with non-specialist policy makers. The findings from these studies exposed the complexities inherent in the production and utilization of policy-relevant knowledge and illuminated issues related to demand, supply, capacity, and intermediation to effectively link supply and demand.⁹

Efforts to apply knowledge-to-policy models in practice, coupled with the adoption of systems thinking, led to conceptions of knowledge generation and utilization as a distinct sector and as an ecosystem.¹⁰ A key concept in systems thinking is that system outcomes emerge from interactions among the four factors listed above and can only be imperfectly identified or predicted in advance. This places a premium on tools such as network analysis to complement stakeholder analysis and political economy analysis. It also suggests that detailed planning and design are unlikely to yield anticipated results. Rather than the traditional project management model of predict-implement-control,¹¹ knowledge-to-policy models apply adaptive management principles that emphasize iterative trial-and-error approaches.¹²

Integrating a systems perspective leads practitioners to focus not simply on the capacities, interests, and roles of the actors in the knowledge system but on the linkages and interactions among them. Evidence for the value and contribution of knowledge to socioeconomic progress and effective policy making has grown (see, for example, Kaur and Singh¹³). The Indonesia case that we explore in this brief builds on the lessons of research and practice briefly summarized here and illustrates the challenges in capitalizing on the potential of knowledge to effectively inform policy.

While many of the programs that RTI implements around the world contribute to various individual elements of a country’s knowledge system, here we focus on a program that incorporates all the elements of knowledge systems previously discussed. The Knowledge Sector Initiative (KSI)^†† RTI has implemented KSI since 2013, in partnership with the Overseas Development Institute, Australian National University, and the Nossal Institute of Global Health at the University of Melbourne, Australia. is a partnership between the Australian and Indonesian governments that seeks to improve public policy by strengthening systems to encourage the use of research and evidence in policy making.³ Developing countries like Indonesia that have achieved middle-income status are increasingly seeing the importance of investing in knowledge systems in an effort to transition to a knowledge economy.¹⁴

As outlined in the 2014 ADB report on knowledge-based economies in Asia,¹⁵ Indonesia scored 3.11 on the knowledge economy index compared with an average 4.39 for countries in Asia and the Pacific and average 8.25 for OECD (Organisation for Economic Co-operation and Development) countries. For Indonesia to successfully transition to a knowledge economy (defined as an economy based on production, distribution, and use of knowledge and information), it will need to move away from extractive industries and natural resources and invest in the knowledge sector including science, technology, and innovation.^16,17 The passage of Law No. 11/2019 on a national system of science and technology in Indonesia provides for a national endowment fund for research as well as a new research and innovation agency tasked with coordinating Indonesia’s research agenda. These are positive steps toward the reforms needed to promote a knowledge economy but are only one part of the overall knowledge system.

RTI has taken lessons learned from the KSI case, as well as from our evidence-to-policy initiatives in other parts of the globe,^§§ Examples include the USAID Strengthening Research, Technology, and Innovation in Development Program (STRIDE) in the Philippines, USAID Higher Education for Economic Growth Program in El Salvador, and the USAID Local Enterprise and Value Chain Enhancement Project (LEVE) in Haiti. and distilled them into five recommendations that are relevant for knowledge systems beyond Indonesia. These recommendations are useful for donor agencies designing new projects related to knowledge systems; practitioners implementing knowledge systems related programs or elements of knowledge systems in sectoral programs; host country government ministries developing knowledge systems strategies or implementing existing strategies; universities, research institutes, or think tanks positioning themselves within a host country knowledge system; and host country civil society organizations and private sector interested in engaging in the knowledge system.

• Promote debate among a diversity of voices within the knowledge systems. Knowledge systems do not consist of mutually exclusive groups of researchers and policy makers encapsulated in neat Venn diagrams. Their reality is much messier, and the richness is precisely in the interactions among the multiple components of the system. In that respect, the greater the diversity of voices interacting, the better. We began KSI neatly distinguishing between researchers, policy makers, and intermediaries. However, as implementation progressed, we observed that often a researcher plays a key role in policy formation, not just knowledge production, and that when policy makers are involved in knowledge production processes from the outset, outcomes are often more robust and inputs more directly applicable. It is important to encourage a broad range of voices interacting in knowledge production and use. Some will be researchers commissioned by policy makers, others will be critical voices of dissent, and still others will be neutral media outlets and such. It is the interaction—the push and pull, the process of challenge and justification—between multiple sources of knowledge that produces rigorous and well-informed policy. Recognizing the power differentials in this push and pull is central to maintain the diversity of voices.

• Provide sustained stakeholder commitment to the systems approach. Knowledge system work is different from, and in fact methodologically contradictory to, most development projects involved in knowledge production. Many development projects provide direct technical assistance, often delivered by foreign experts embedded within ministries, or through analytics and research paid for by the project at the direct demand of policy makers. Building a country’s knowledge system, on the other hand, requires the slow, often painstaking work of building the systems (structures, regulations, budgets, and policies) to deliver these analytics without reliance on donors or foreign expertise. This is usually a deeply political effort of aligning interests, competing over resources, and building the sustainability and capacity of local research institutes. It is not a quick fix, nor does it always provide immediate policy impact. It requires patience, visionary leadership, and commitment from key stakeholders within the political apparatus. The following textbox illustrates the differences in the knowledge systems approach.

• Invest in knowledge system components as well as the interaction between components. A systems approach may need to be introduced gradually in new country contexts where the knowledge sector is still weak or nascent by investing in subcomponents of the knowledge system (supply, demand, intermediary, enabling) to build a foundation for interaction among stakeholders. Some donors may invest in one or more subcomponents of knowledge systems such as

• improved research quality, research communication, or institutional development of research centers and think tanks (supply);

• support to government ministries on identifying a research agenda, commissioning research, or extracting research findings and translating them into policy inputs (demand);

• resources for independent media on investigative journalism for policy, evidence-based advocacy training for civil society organizations (intermediaries); or

• analytics related to regulatory reform and support for multi-stakeholder groups on new approaches to research funding (enablers).

• It may take time to build the fundamentals of evidence-to-policy in each of these components before constructive interaction will start to bear fruit. In countries with highly functioning knowledge systems, key stakeholders in government, the research community, and civil society cooperate to manage all of these components as well as the interactions among them. These interactions build commitment to a national innovation or knowledge systems plan that outlines joint ideas, goals, and targets. Establishing a coordinating agency facilitates collaboration between different organizations during the implementation of the national plan and mitigates any turf wars.¹⁴

• Foster a balance between government-led mechanisms and space for civil society perspectives. Sustainability of the knowledge sector depends upon investment in government-led governance and systems. However, providing spaces for independent voices and challenges from civil society is essential, both for influencing emerging policy agendas and for offering input to existing policies. The advantage of government-led governance and systems is ownership from policy makers who are directly involved in shaping discourse and responsible for promoting evidence-based policy within their ministries or line agencies. However, in establishing national priorities it is important to include spaces for civil society to raise issues that may not yet be recognized by government or minimized due to political reasons. Here again, attention to power dynamics is key.

• Explore how knowledge systems can engage the private sector. In knowledge systems, the private sector has not commonly been recognized as a major player, but that is changing. Private foundations and corporations are increasingly important funders, producers, and consumers of research for policy purposes, and they engage in knowledge systems in ways ranging from endowing university research chairs and funding scholarships to direct lobbying activities. The Gates Foundation, for example, supports research on HIV globally that is used to influence health policy in multiple countries.

The recommendations above summarize the principles in this policy brief related to successful knowledge systems development. Their emphasis on diversity of knowledge, recognition of the multiple and overlapping roles of knowledge system actors, and the highlighting of power dynamics in knowledge systems interactions deepen understanding of effective knowledge systems. For application in new country contexts, we recommend a political economy analysis be undertaken to determine areas with the largest potential for reform. Preconditions useful to ensure successful implementation include some degree of acknowledgment of the importance of evidence-based policy making and commitment to doing it (this does not need to be widespread, but change agents with influence somewhere within government are needed).

In addition, some researchers and associated institutions need to be willing to commit to policy research and policy engagement (not all research institutes or universities research centers may be interested, but a critical mass is needed). Finally, press freedom is needed to enable the provision of contrasting evidence or challenges to current policies (again, this does not need to be widespread but at least should offer an opening for presenting evidence that may stray from the status quo). These various facilitating elements can be supported in different ways to strengthen the foundation for knowledge system development and institutionalization.

These preconditions are not essential for knowledge systems to grow, but where they are absent, additional awareness-raising may be needed regarding the importance of evidence-based policy making and how it contributes to a knowledge economy. By demonstrating the benefits of the approach and taking small steps to test the process, the number of committed individuals and institutions can grow, as the KSI experience demonstrates. KSI spent significant time in the early years focused on raising awareness about what knowledge-to-policy means and how it can make a difference. Given the global threats of fake news, hoaxes, and populist distrust of expertise, robust knowledge systems continue to be relevant. This research brief offers some insights on how to develop and sustain them.

The authors gratefully acknowledge the financial support from an RTI independent research and development grant for this publication. We also extend our appreciation to the Australian Department of Foreign Affairs and Trade-(DFAT) funded Knowledge Sector Initiative team for support of knowledge systems reforms highlighted in this brief. We would also like to acknowledge the background research conducted by Kara Reeve; helpful comments from Eric Johnson, Anna Wetterberg, Sara Lawrence, and Budiati Prasetiamartati; constructive comments from the anonymous reviewer; fact checks from Irene Pingkan Umboh and Sugiyanto; graphics support from Sonja Douglas; and photo support from Benedicta Kusuma.

Cover photo credit: Hubertus Gorat, Knowledge Sector Initiative

RTI Press Associate Editor: Elizabeth Glennie