What Have We Achieved? A Pragmatic Guide to Assessing Policy Engagement
Key features in a nutshell
- No One-Size-Fits-All: Assessment criteria must match the scope, duration, and goals of your specific engagement activity; a single breakfast talk requires a different evaluation than a multiyear expert commission
- Planning is key: Understanding the effects of a format is much more feasible once it is strategically agreed upon beforehand what effects the format is desired to have
- Understand effects beyond policy change: Direct legislative change is rare and multicausal; process indicators – such as relationship building, follow-up requests, and mutual learning – are often more realistic measures of success
- The right combination of resources and focus: There are almost endless possibilities when assessing one’s project but usually limited resources; to make the process and the results useful, it’s best to focus on specific aspects or questions that help projects grow or change and that match the resources of the project (e.g., to collect and assess data)
What is the issue?
How can researchers and research managers better understand the effects of their policy engagement activities? This is highly relevant for policy engagement activities to justify the time and financial resources invested, to demonstrate accountability to funders, and – most importantly – to reflect on and improve future interactions. It can also enable better alignment between the needs of policymakers and what advice science can provide.
However, as discussed in our article on Policy Engagement Goals, success looks vastly different depending on what you set out to achieve. External factors such as political timing, shifting government priorities, resource constraints, or sheer luck heavily influence outcomes as well.
A multiyear advisory board presenting a high-profile report operates in a different reality than a research team hosting an introductory lunch briefing on an emerging topic. Therefore, understanding different effects is not a standardised checklist but a flexible instrument, which starts with agreeing on goals and desired effects .
Possible goals of policy engagement can:
- Policy change: shaping how challenges are framed, approached, and dealt with (e.g., legislation, directives),
- Raising visibility, building trust, and expanding networks
- Mutual learning: understanding policymaking, and gaining ideas for your own research
- Open-ended informing of policymakers
Four levels of effects criteria
Different effects levels can be roughly identified, depending on the objectives and goals of the policy engagement activity as well as the resources available to examine them. It should be noted that exploring these effects can be a time-intensive activity in itself, which may require some expertise. For researchers and knowledge mobilisers, it can therefore be helpful to team up with experienced researchers or practitioners if possible.
Here we name four distinct levels of effects
Level 1: Reach and participant composition (input & output)
- What it measures: Who attended your activity and in what capacity?
- Key indicators: Number of participants, institutional diversity (e.g., representation from ministries, parliaments, NGOs, or think tanks), and the fit of the policymakers (i.e., the policymakers dealing with the policy issues)
- When should it be used: Ideal for initial, low-cost formats (e.g., public panel discussions, introductory webinars, or policy breakfasts)
Level 2: Interaction quality and relationship building (process)
- What it measures: Was the interaction perceived as valuable, and did it result in follow-up activities?
- Key indicators:
- Qualitative/quantitative feedback: Surveys (e.g., via the Impact Unit Evaluation Platform) or short debriefing interviews asking both researchers and policymakers whether the activity was rewarding and informative
- Follow-up activities: Unsolicited requests from policymakers after the event, organisation of joint events, informal follow-up conversation
- When should it be used: Essential for workshops, roundtables, and ongoing dialogue series aimed at long-term network building.
Level 3: Referencing and document citations (uptake)
- What it measures: Did your research findings inform policy discussions or official publications?
- Key indicators: Policymakers reference the findings or recommendations of individual researchers or research groups in media statements, press releases, or speeches; citations of academic publications or policy publications by researchers in policy documents like strategy papers, parliamentary inquiries, party platforms, or policy reports
- Tools: Policy citation databases such as Overton, Dimensions, or PlumX make it easy to track where academic work is cited in policy literature worldwide
- Caveat: Being cited in official policy documents indicates visibility and engagement, but it does not automatically mean that a policy was altered; note also that policy citation rates vary heavily across disciplines (e.g., public health and economics yield far higher citation counts than cultural studies)
Level 4: Attributable policy change (direct impact)
- What it measures: Did the scientific evidence directly shape a policy decision, legislation, or government programme?
- Key indicators: Explicit statements by decision makers in legal texts, parliamentary speeches, press releases, or official memoirs confirming that scientific advice influenced a specific decision
- Caveat: Attributable policy change is the highest bar and exceptionally rare; research is almost always one of many competing inputs in the policy process; expecting every engagement effort to yield direct policy change is unrealistic and leads to frustration
Tasks and resources
Researchers
Researchers are well positioned to reflect on how the interaction felt from their own perspective, if it resulted in sustainable connections and advanced their research. They are also a valuable source when it comes to understanding demands or challenges policymakers face by looking at questions that were asked, concerns that were raised, or topics that were (controversially) debated.
Research managers and knowledge mobilisers
Research managers typically design and distribute feedback questionnaires and track key data like number of participants, downloads, or follow-up contact between researchers and policymakers over time. They can also support researchers in documenting instances in which policy change is attributable to their work by, for example, monitoring government publications, coordinating with communications teams, or interviewing policymakers and their teams. Furthermore, (participant) observations or question-based protocols are helpful in documenting and assessing specific interactions or formats.
Policymakers
Policymakers contribute primarily through feedback and through their own public or written statements. Their input is most valuable when it is solicited directly and specifically through, for example, short structured questionnaires after an event or brief follow-up conversations, rather than assumed from silence or general goodwill.
Sources and further reading
Impact Unit (2026), Evaluation platform, https://evaluationsplattform.impactunit.de/, accessed on 20 August 2026.
Neicu, D., Cauchi, J., Otto, J., Lehto, S., & Dantas Faria, J. C. (2020), Monitoring the Impact of Science and Evidence on Policy, in Šucha, V. & Sienkiewicz, M. (eds.), Science for Policy Handbook, Elsevier, pp. 152–164. https://doi.org/10.1016/B978-0-12-822596-7.00014-0
Newson, R., King, L., Rychetnik, L., Milat, A., & Bauman, A. (2018), Looking Both Ways: A Review of Methods for Assessing Research Impacts on Policy and the Policy Utilisation of Research. Health Research Policy and Systems, 16, Article 54. https://doi.org/10.1186/s12961-018-0310-4
Overton (2026), Tracking Research Impact in Policy Documents. [Online Resource], accessed on 14 August 2026.
Reed, M. S. (2022), The Research Impact Handbook: Generating and Assessing Impact from Research. Fast Track Impact.
Do you want to know more about the policy engagement toolkit?
Do you want to share your experience in policy engagement? Do you think about contributing to the policy engagement toolkit? Feel free to contact Tome Sandevski via science-policy@pvw.uni-frankfurt.de
