Sažetak – engleski

 

 

 

SUMMARY OF MONITORING AND EVALUATION FINDINGS
TECHNICAL SUPPORT TO THE IMPLEMENTATION
OF THE COMPREHENSIVE CURRICULAR REFORM IN
CROATIA

 

 

Summary of Monitoring and Evaluation Findings

An introduction by Artur Taevere, Principal Consultant at British Council and technical lead of the project.

1. Phase 1 of our project (from March 2018 to March 2019) has made a significant contribution to key aspects of the Comprehensive Curriculum Reform in Croatia. We focused on four themes: (1) learning outcomes, (2) problem solving, (3) lea1rning to learn, and (4) coaching. Over 1,000 teachers and principals from experimental schools responded to our survey in February 2019, conducted by the University of Auckland. A significant majority indicated that they understand these key changes and already practice them or intend to practice them in the future. Notably, 81% of respondents indicated they understand what learning outcomes are, 74% already use them in practice and 84% intend to use them in the future. With problem solving and learning to learn, somewhat fewer teachers already use these approaches (69% and 59%, respectively), but a high proportion of respondents intend to use them in the future (83% and 81%). This suggests that teachers and principals have been motivated to learn about these new approaches and are beginning to improve their practice accordingly. Detailed figures are available on page 36 of the evaluation report by Claire Sinnema from the University of Auckland.

2. Our project has contributed to the smooth and effective implementation of the pilot phase of the curriculum reform mainly by strengthening the capacity of Croatian mentors. Mentors are experienced teachers and educators, released from their daily teaching responsibilities, to support teachers in experimental schools who are engaging with the new curriculum. After the training programme conducted by international experts in Sveti Martin in May 2018, 100% of mentors agreed that they are better prepared to support teachers in experimental schools. 80% agreed with this statement either strongly or very strongly. The learning from Sveti Martin has been extended by virtual webinars over the past six months, which has further prepared mentors to support teachers in experimental schools.

3. There is broad based support among teachers, principals, parents and students for the curriculum reform. For example, 85% of principals and teachers, and 76% or parents were either fully or mostly supportive of the new curriculum (Sinnema, pages 9 and 11). This broad support has been achieved because of many reasons, including our project, but also many other initiatives and factors. Many teachers feel that Croatian schools have been ready for change. In a focus group conducted in October 2018, a teacher said: ‘I would like to compliment the fact that, despite everything, something is starting to change. These are systemic changes. These are not cosmetic changes’ (Priestley, page 11). In our survey of teachers and principals, most of them also have positive views about the quality of support they have received. Numerous initiatives by the Ministry of Science and Education, and others, have contributed to these results. Subject specific training has been conducted for teachers, regional training events held on various topics, experimental schools have been provided with technology such as tablets, new textbooks have been procured, etc.

4. However, the positive results about support for the curriculum reform and openness to change teaching practices should not be over-interpreted. On the one hand, more than two thirds of survey respondents indicated that they have made moderate or substantial change to how they use resources in teaching, and the sort of activities they used in classes. More than half indicated they have made moderate or substantial changes to planning documentation, the role students take in their class, their interactions with students, content, or topics used (Sinnema, page 10). While these results are encouraging, it is important to note that these are based on self-reporting. In the next phase of the project, it will be useful to add classroom observations as a further strand to monitoring and evaluation. This would give us further information to what extent the changes report by teachers and principals can be observed in classrooms.

5. Making sense of the new curriculum takes time. It takes time to understand the new ideas and practices, try them out, and reflect on how they work. Claire Sinnema writes in her report: ‘Well-intentioned practitioners often over-report the progress they’ve made, given the likelihood that new ideas have not been fully understood’ (Sinnema, page 10). Mark Priestley makes a similar point in his report: ‘…because this new curriculum will be interpreted through the extant knowledge, beliefs, and experiences of the individual, the act of making sense of the new information is always mediated by combining the existing understanding with the details of the new curriculum. As such, the teachers often interpret the reform ideas as being aligned with their existing ones. Thus, the new information, instead of supplanting the existing knowledge and practice, merely supplements it. This creates what Spillane (2004) refers to as a short-cutting process, which means that, while the teachers are enthusiastic about the reform, their enactments of the new curriculum are often superficial, thus overlooking the deeper structural or conceptual changes encompassed in the reform (Priestley, page 4).

6. Continuing to invest in the professional development of teachers, principals and other school staff members will enable the curriculum reform to succeed in the long term. As the monitoring and evaluation reports highlight, the reform currently enjoys broad based support among teachers, principals and families. The good work simply needs to continue. Professor Graeme Aitken from the University of Auckland has summarised the key ideas of the Croatian curriculum reform as: (1) changing the focus from teaching to learning; (2) changing the learning purpose from the acquisition of knowledge to the development of competencies and values alongside knowledge; (3) changing the teaching approach from teaching as telling to teaching as actively engaging students through using a range of motivating methods relevant to outcomes; (4) changing the learning approach from providing students with content to helping students learn how to learn. These ideas of the comprehensive curriculum reform can be realised, but only if teachers, principals and other key staff members in schools continue to receive high quality support – helping them understand the deeper meaning of the new curriculum and how to make it happen.

The following reports are included:

1. Curriculum Reform in Croatia: Monitoring and Evaluation Report

Part 1: Evaluation Survey (Summary of quantitative research)
By Associate Professor Claire Sinnema with support from Justine Park, The University of Auckland
The report evaluates teachers’, principals’ and parents’ response to the curriculum reform in Croatia and their understanding of the new curriculum. Two surveys were administered in February / March 2019, one for principals and teachers, and the other for parents. The survey of principals and teachers primarily focused on the areas of the curriculum that were addressed in Phase 1 of our project: learning outcomes, problem solving, learning to learn and coaching. The survey of parents considered their general understanding and views of the curriculum reform.

2. Curriculum Reform in Croatia: Monitoring and Evaluation Report

Part 2: Theoretical Framework, Curriculum Narratives, Commentary on Curriculum Framework and Review of Science and Social Studies Curriculum (Summary of theoretical review of the curriculum reform)
By Professor Graeme Aitken, The University of Auckland
The Theoretical Framework describes curriculum implementation as a process whereby teachers genuinely try to make sense of what policy is requiring of them. They need to be supported by well-designed professional development that helps them recognise the deeper conceptual understandings of the reform, engages their prior knowledge, balances the general and specific.
The Curriculum Narratives suggests a process of making sense of the key elements of the reform and what they mean in practice.
Commentary on Croatian National Curriculum Framework Documents provides a brief theoretical review of the National Curriculum Framework, the Evaluation Framework and the inter-subject topic Learning to Learn.
Finally, the Science and Social Studies curriculum is reviewed in terms of appropriateness, relevance, breadth and depth, etc.

3. An Exploration of Curriculum Reform in Croatia

Summary of qualitative research
Professor Mark Priestley and Aileen Ireland, The University of Stirling
This report contributes to the overall evaluation of our project in Phase 1, drawing upon qualitative research undertaken in five experimental schools in Croatia in October 2018. Data was collected from interviews and focus groups with principals, teachers, students and parents. The interviews and focus groups covered the curriculum reform more broadly, not just the four themes that our project focused on in Phase 1 (learning outcomes, problem solving, learning to learn, coaching). The feedback and comments are not specifically about our project, but about the general experience of engaging with the new curriculum and various projects related to it.


Curriculum Reform in Croatia

Monitoring and Evaluation Report
for the Technical Support to the Implementation of Comprehensive Curricular Reform in Croatia

Part 1: Evaluation Survey

Prepared by
Associate Professor Claire Sinnema,
with support from Justine Park
School of Learning, Development and Professional Practice
at the Faculty of Education and Social Work,
The University of Auckland

 

Project Context

The British council has been implementing a project of ‘Technical Support to the Implementation of Curricular Reform’ in Croatia funded by the European Union via the Structural Support Service of the Commission. The comprehensive curricular reform is a wide-ranging initiative led by the Ministry of Science and Education in Croatia. The overarching objectives of the comprehensive curriculum reform are about the establishment of a harmonised and effective education system based on comprehensive curriculum and structural changes in order to be able to:

Many projects and programmes contribute to the experimentation program for the reform, including the support project implemented by the British Council that is the focus of this evaluation. The overall objectives of the support project are:

The technical support has been provided to those taking part in the experimental program which aims to check the applicability of new curricula and focuses on goals relating to a) increasing student competencies in solving problems and b) Increasing students’ satisfaction with the school and motivating their teachers and teachers.

 

Purpose of the Evaluation

The purpose of this project was to evaluate teachers’, principals’ and parents’ response to the curriculum reform in Croatia and their understanding of the new curriculum. The scope of the evaluation focused on key elements of the National Curriculum Framework (NCF) which is the main national curriculum document (Ministry of Science and Education, 2017) and defines the following:

 

Key Evaluation Questions

The evaluation was designed to provide insights into the following key evaluation questions:

  1. To what extent are teachers and principals involved in the support programme receptive to the new curriculum in Croatia?
  2. How confident are teachers and principals to give effect to the new curriculum in Croatia?
  3. How do teachers and principals understand key aspects of the new curriculum in Croatia?
  4. To what extent are teachers and principals giving effect to key elements of the new curriculum in their practice?
  5. To what extent are parents aware of and supportive of the new curriculum in Croatia?

Summary of Methodology

Carried out in February/March of 2019, this survey drew on data from teachers, mentors, principals and parents. Two surveys, one for principals/teachers and the other for parents used a range of agreement scales and semantic differential scales to capture perceptions in relation to

 

Sample and data sources

 

Principals’/Teachers’ questionnaire: A total of 1624 teachers and principals were invited to participate. A total of 1281 respondents (79% of those invited) gave consent and began the online principal/teacher survey. Of those who began the survey, 1047 (82%) completed in full representing a response rate from the original sample invited of 64%. Full completion of all substantive items of the survey in involved responses to all items on the following scales: overall support, receptiveness, support, curriculum elements and curriculum realisation, and change). The remaining partial responses, the vast majority of which were very partial (only 1 or two items or scales completed) were removed for this analyses.

The principals/teachers dataset of complete responses comprised 45 who indicated that they were principals, 22 indicated that they were mentors, 891 indicated that they were teachers, and 121 indicated they held an ‘other’ role1.

Responses were received from 47 primary schools (with an average of 15 respondents per school) and from 27 secondary schools (with an average of 17 respondents per school).2
Parents’ survey: A total of 1335 respondents completed the online parent survey. A further 152 opened the survey but did not complete any responses so were removed from the dataset.

Summary of Key Findings – Principals’/Teachers’ Survey

Principals’ and teachers’ overall support

In response to the question ‘Overall, how supportive are you of the new curriculum?’, overwhelmingly respondents were supportive. More than 85% or all respondents were either fully or mostly supportive of the new curriculum. Notably, Mentors all responded in the supportive (either little, mostly or fully) response options with more than 95% either fully or mostly supportive. Principals were also strongly supportive with the highest number of any group indicating full support (51%) and more than 95% either fully or mostly supportive.

 

Principals’ and teachers’ receptiveness to the new curriculum

Responses to the questions about how they view the curriculum, designed to indicate their receptiveness, were also highly positive. Findings from semantic differential items (with a more negative view indicated at one end of a six-point scale, and a more positive view indicated at the other) revealed that only one item had fewer than half responding in the positive end of the continuum (one of the three positive response options)—that item was the one about their view about whether the new curriculum required  more work (than the previous curriculum)’ or ‘less work (than the previous curriculum)’. This suggests that practitioners are mindful of the workload involved in realising the aspirations they are highly receptive to.

Principals’ and teachers’ views of support quality

Approximately two thirds of Principals and teachers’ indicated, on average across all support quality items, positive views of support quality (a response in one of the three positive points of the continuum). The most highly rated items were those focused on whether support connected with pre-existing beliefs, was a source of new ideas and insights, influenced practice, and was challenging. The lowest rated items were those asking for respondents for their views about how generous, sound and relevant the support was.

Principals’ and teachers’ practice for key curriculum elements – curriculum realisation

Responses to questions about eight key curriculum elements and the extent to which principals and teachers understand the reasons for them, what they are, think they are a good idea, are confident in using them, have colleagues with expertise to turn to for help, use the element in practice already of intend to suggest that respondents consider themselves to be progressing well in enacting the new curriculum. These findings should be interpreted in caution, since studies in other contexts indicate that policy enactment efforts even by well-intentioned practitioners may be less well-progressed than respondents report themselves given the likelihood of new curriculum ideas not yet being fully understood or implications fully realised.

Change

More than two thirds of respondents indicated that they have made either moderate or substantial change to resources used, and activities used. More than half (but fewer than two-thirds) of the respondents indicated that they have made either moderate or substantial change to their planning documentation, the role their students take in class, their interactions with students and content or topics used. Slightly fewer than half of respondents indicated that they have made either moderate or substantial change to their interactions with colleagues and their approach to reporting to parents.

Cross-curriculum topics

Respondents ranked in order those cross-curriculum topics they are most likely to embed (1) to those they are least likely to embed (7). Civic education was the cross-curriculum topic most frequently ranked first suggesting practitioners are prioritising this topic. Entrepreneurship and use of information and communication technologies were the cross-curriculum topics least likely to be highly ranked.

Generic competencies

Principal and teacher respondents were asked to indicate which of the generic competencies they are likely to most effectively focus on in their practice (teaching or leadership). Respondents ranked in order those cross-curriculum topics they are most likely to focus on (1) to those they are least likely to focus on (7).
Forms of thinking (Resolving problems; Making decisions; Metacognition; Critical thinking; Creativity and innovation) was ranked first by more than twice as many respondents as the other generic competencies, with the forms of work and tools generic competency (Communication; Cooperation; Information literacy; Digital literacy and the use of technologies) ranked lowest by most respondents.

Correlations

  • Pearson correlations for the following seven scales related to the theoretical framework in the evaluation design are outlined in the table below.
  • Support quality
  • Regard
  • Confidence
  • Network
  • Understanding
  • Practice—design
  • Practice—teaching

The correlation analyses above suggest that:

  • There are statistically significant correlations between all scales
  • Support quality is moderately correlated with both regard and confidence
  • Regard is strongly correlated with confidence and understanding
  • Regard is moderately correlated with network and practice – design
  • Confidence is strongly correlated with understanding and practice – teaching
  • Network is moderately correlated with confidence and practice – teaching
  •  Understanding is strongly correlated with regard, confidence and practice – teaching
  • All other correlations are small but statistically significant

 

Summary of Key Findings – Parents’ Survey

Parents’ overall support

Responses to the parents’ survey indicate high levels of support for the new curriculum. More than 76% indicated that they are either mostly or fully supportive of the new curriculum, and very few (5%) indicated that they are not supportive.

Parents’ views of the new curriculum

The majority of respondents to the parents’ survey were more positive than negative leaning (rating one of the three points at the positive end of the semantic differential scale) about the curriculum being necessary (77%), about their positive view of the curriculum (82%), about the curriculum being better than the previous one for their child (73%) and about viewing the curriculum as exciting (77%).

Parents’ awareness, understanding and support of the new curriculum

The majority of respondents to the parents’ survey indicate strong agreement that they are aware of the new curriculum and of their child’s school participating in the experimental programme.

Parents’ understanding and support of the new curriculum

There were high levels of agreement from respondents to the parents’ survey regarding support of key curriculum elements (a focus on learners’ problem-solving, a focus on cross-curriculum topics, and a focus on generic competencies). More than two thirds of respondents strongly or very strongly agreed
that they support those foci—problem-solving (74%), cross-curriculum topics (73%), students’ active role (68%). Fewer indicated that they strongly or very strongly agreed that they understand the purpose of the new curriculum (53%) or understand the key changes (44%).

Conclusion

This evaluation suggests that, like in other settings, practitioners have high regard for the new curriculum they are charged with realising for the children and young people they are responsible for. With high support quality, teachers and leaders can build the confidence required to give effect to the curriculum—that requires strong network, and deep understandings of the new curriculum elements in particular. Theses analyses only touched on self-report accounts of understanding, and so is a limitation of the study. Further analyses of the understandings teachers, principals and mentors bring to bear as they bring the curriculum to life in classrooms (and as they respond to evaluative surveys) are important to consider alongside these quantitative findings. Subsequent analyses will draw on qualitative responses for insights into both perceptions and understandings of the curriculum.
Respondents to the parents’ survey were also highly positive in terms of overall support, their views about the quality of the curriculum and their support of key elements.
These early positive indications bode well, but do not mitigate the need for ongoing efforts: to continue to provide high quality support; and to strengthen the networks through which curriculum expertise is spread. These efforts will be important in order to retain the high regard for the curriculum and build the confidence and capability required to enact it in ways in line with the intentions and aspirations set out in the curriculum.

 

 

_____________

1 Subsequent analyses will examine text entries for the ‘other’ category which may result in revisions to the frequency counts for principal, teacher and mentor categories
2 Subsequent analyses will examine text entries for the ‘other’ responses to the name of school item which may result in revisions to the frequency counts for schools and the average number of respondents per school type

 

The project is funded by the European Union via the Structural Reform Support Programme and implemented in cooperation with the European Commission’s Structural Reform Support Service