Evaluating the Impact of Local Science-Policy Dialogue Events

A Methodological Approach from the UGR County Councils Initiative

Summary: Key features in a nutshell

The evaluation instrument was designed with the following objectives:

• Measure the satisfaction of attendees across the four sectors of the quadruple helix
• Assess the perceived quality of networking and knowledge exchange
• Evaluate the understanding and perceived usefulness of the UGR County Councils initiative
• Capture differences in perception across academic and administrative profiles
• Identify actionable recommendations for improving inclusivity, policy relevance, and follow-up mechanisms in future living lab–based encounters

Description

The UGR County Councils (Consejos Comarcales UGR) initiative, part of the Erasmus+ ENGAGEgreen project, aims to promote structured science-policy dialogues across the ten counties of the province of Granada. In July 2025, the first territorial encounter brought together researchers, policymakers and civil society representatives in the historical Azucarera of Motril. To assess the impact of this event and strengthen future participatory formats, the University of Granada developed an evaluation survey specifically tailored to the quadruple helix model and the regional context.

This methodological approach was tested during two major events — the First Impronta Granada Meeting and the First Meeting of UGR County Councils — allowing comparison of perceptions and outcomes across different regional settings

How does the format work?

The survey was administered online using Google Forms and disseminated directly after the event. Participants were invited via email and QR codes distributed during the meeting. The form consisted of nine core questions divided into thematic blocks:

1. Identification block: sector affiliation (Academia, Public Administration, Business, Civil Society)
2. General satisfaction block: Likert scale (1 to 5) evaluating the overall meeting, networking opportunities, and event logistics. The event logistics was analyzed with four Likert-scale questions on the location, schedule, content, and facilitation
3. Participatory dynamics block: Items assessed inclusivity, perceived listening, and sense of contribution to the dialogue
4. Specific feedback blocks tailored to– Academic participants: changes in understanding of policy processes, interest in continued engagement– Public administration participants: prior experience with research, usefulness of data shared, intent to maintain contact.
5. Open feedback block: Suggestions for future events with optional contact details.

The evaluation process combined quantitative and qualitative approaches, including participant observation and informal feedback during networking sessions, to triangulate survey results and deepen the analysis of engagement dynamics.

Challenges

The questionnaire was developed by the technical coordination team of the Erasmus+ ENGAGEgreen project at the University of Granada. No external resources were required beyond institutional communication channels and Google Workspace tools. Internal collaboration with the data protection office ensured the compliance of the Regulation on the Protection of Personal Data of Individuals.

The main methodological challenge was balancing brevity with analytical depth, as short online forms sometimes limited the detail of qualitative responses.

Tasks and resources

The differentiated blocks for academics and policymakers proved effective in capturing specific expectations and experiences. The use of Likert scales allowed for a quantitative analysis of perceived impact, while the open section provided rich qualitative insights. Most notably:

• High levels of satisfaction and expressed interest in continued collaboration
• Strong appreciation for the opportunity to exchange knowledge in a regional context
• The methodology facilitated the identification of gaps between policy needs and academic communication strategies
• Tailored insight into institutional and individual motivations
• Better understanding of perceived usefulness of the event
• High levels of satisfaction with logistics, participation equity, and perceived listening
• A growing demand for more frequent formats of pairing and localized science-policy interaction.

Conclusion

Evaluating science-policy engagement formats is essential to ensure their effectiveness and replicability. The UGR evaluation model demonstrates how local events rooted in territorial identity can be assessed through simple but robust methodologies that reflect both institutional goals and participant experiences.

Scientists

• Designed and validated the mixed-methods evaluation protocol ensuring scientific robustness while maintaining accessibility for non-academic participants
• Analzsed quantitative data from 43 and 21 respondents across two events, identifying patterns of satisfaction and collaboration intentions
• Interpreted qualitative feedback to detect emerging themes such as commitment to collaboration, clarity of objectives, and need for stronger political engagement
• Produced evidence-based recommendations to enhance time allocation, venue accessibility, and follow-up strategies, thus reinforcing the Living Lab methodology’s policy impact
• Reflected on methodological transferability to other regional science-policy dialogues under ENGAGEgreen

Knowledge exchange managers and research managers

• Coordinated survey dissemination via digital and in-person channels (QR codes, email lists, feedback sessions)
• Conducted participant observation during group work to assess cross-sector dynamics and inclusivity.
• Collected and synthesized informal verbal feedback to complement survey findings
• Managed anonymization and GDPR compliance with the university’s Data Protection Office
• Compiled comprehensive reports integrating both quantitative results (e.g., satisfaction 4.53/5; networking 4.36/5) and qualitative insights, providing an internal learning document for ENGAGEgreen WP3

Policymakers

• Contributed perspectives on institutional needs and constraints through survey responses and interviews
• Used findings to adjust UGR County Councils’ agendas—prioritizing clearer communication of objectives, longer work sessions, and broader political representation
• Recognised the value of systematic evaluation as a tool for evidence-based policy improvement and enhanced collaboration with academia
• Expressed high willingness (avg. 4.50/5) to continue participating in future knowledge-exchange activities

Case studies

How is the format carried out?

Post-event, participants received a digital survey accessible via QR code and email. Data were exported to Excel for quantitative analysis and thematically coded for qualitative insights. Results were discussed internally by UGR’s ENGAGEgreen coordination team to identify lessons and integrate them into the Toolkit.

Available resources (what are the main tasks)

• Survey templates (Google Forms)
• Coding framework for qualitative data
• Feedback report template (summary + recommendations)
• GDPR consent and anonymisation guidelines

What works? What doesn`t? (Lessons learnt)

• Works: concise digital tools; mixed quantitative-qualitative design; immediate post-event collection
• Doesn’t: overly short group sessions; lack of rotation between discussion tables; vague follow-up procedures
• Lessons learnt: always combine survey data with observation notes; report results back to participants to maintain trust

How are the activities evaluated?

• Outputs: Completed feedback reports per event; consolidated WP3 evaluation chapter
• Outcomes: Evidence of increased collaboration intent (4.7/5); improved design of next County Councils
• Indicators: Satisfaction, networking, clarity, inclusivity
• Timing: Immediate post-event collection + synthesis within 2 weeks

Do you want to know more about the policy engagement toolkit?

Do you want to share your experience in policy engagement? Do you think about contributing to the policy engagement toolkit? Feel free to contact Tome Sandevski via science-policy@pvw.uni-frankfurt.de