How can you measure the effectiveness of HE curriculum design for research projects?

2 Approaches to evaluation

Another challenge of evaluating HE curriculum design for research projects is the selection of an appropriate and feasible approach to evaluation. There are different types of evaluation, such as formative, summative, process, or impact evaluation, that serve different purposes and require different methods and data sources. For example, formative evaluation aims to provide feedback and improve curriculum design during its implementation, while summative evaluation aims to assess the overall effectiveness and outcomes of curriculum design after its completion. Similarly, process evaluation focuses on the inputs, activities, and outputs of curriculum design, while impact evaluation focuses on the outcomes, effects, and benefits of curriculum design.

4 Criteria for evaluation

A fourth challenge of evaluating HE curriculum design for research projects is the establishment and application of relevant and meaningful criteria for evaluation. Criteria for evaluation can be derived from different sources, such as the curriculum objectives, the research standards, the student expectations, or the stakeholder feedback. However, criteria for evaluation should also be aligned with the evaluation approach, the tools, and the data analysis. For example, if the evaluation approach is formative and process-oriented, the criteria for evaluation should reflect the quality and adequacy of the curriculum design elements, such as the learning outcomes, the content, the pedagogy, or the assessment. On the other hand, if the evaluation approach is summative and impact-oriented, the criteria for evaluation should reflect the achievement and satisfaction of the research outcomes, such as the knowledge, skills, competencies, or attitudes.

5 Examples of evaluation

To illustrate how the challenges, approaches, tools, and criteria for evaluating HE curriculum design for research projects can be addressed in practice, here are some examples of evaluation from different disciplines and contexts. In a study by Jones et al. (2017), the authors evaluated the impact of a curriculum redesign that integrated research-based learning across all undergraduate programmes in a UK university. They used a mixed-methods approach that combined surveys, interviews, and document analysis, and they applied criteria based on the HEA Research-Teaching Nexus Framework. They found that the curriculum redesign had a positive effect on the students' research skills, confidence, and engagement, as well as on the staff's research-teaching linkages and professional development. In another study by Elmgren et al. (2019), the authors evaluated the process and outcomes of a curriculum innovation that introduced interdisciplinary research projects for master's students in a Swedish university. They used a qualitative approach that involved focus groups, observations, and reflective journals, and they applied criteria based on the CDIO (Conceive, Design, Implement, Operate) model. They found that the curriculum innovation enhanced the students' interdisciplinary collaboration, communication, and problem-solving skills, as well as their awareness of the societal relevance of their research projects.

6 Implications for practice

Evaluating HE curriculum design for research projects is a complex and context-specific undertaking that requires careful planning, implementation, and interpretation. However, it can also provide valuable insights and feedback for improving the quality and relevance of HE teaching and learning, as well as for enhancing the research culture and capacity of HE institutions. Therefore, HE practitioners should define the purpose and scope of the evaluation to align with curriculum objectives and research standards. Additionally, an appropriate and feasible evaluation approach should be selected, using a combination of quantitative and qualitative tools that suit the evaluation questions and data sources. Establishing and applying relevant criteria for evaluation is essential to guide data analysis and reporting. Involving key stakeholders, such as students, staff, or external partners, throughout the process is important to seek their input and feedback. Ultimately, the evaluation results should be used to inform and improve the curriculum design and implementation, as well as to disseminate and celebrate the research outcomes and achievements.