《經合組織（OECD）：2024芬蘭教育評估中心(FINEEC)審查報告（英文版）（79頁）.pdf》由會員分享，可在線閱讀，更多相關《經合組織（OECD）：2024芬蘭教育評估中心(FINEEC)審查報告（英文版）（79頁）.pdf（79頁珍藏版）》請在三個皮匠報告上搜索。

1、OECD Skills StudiesFinnish Education Evaluation Centre(FINEEC)OECD CENTRE FORSKILLS EVALUATIONSFinnish Education Evaluation Centre(FINEEC)OECD CENTRE FORSKILLS EVALUATIONSOECD Skills StudiesOECD Skills StudiesFinnish Education Evaluation Centre(FINEEC)OECD CENTRE FOR SKILLS EVALUATIONSThis work is p

2、ublished under the responsibility of the Secretary-General of the OECD.The opinions expressed andarguments employed herein do not necessarily reflect the official views of the Member countries of the OECD.This document,as well as any data and map included herein,are without prejudice to the status o

3、f or sovereignty overany territory,to the delimitation of international frontiers and boundaries and to the name of any territory,city or area.The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities.The use ofsuch data by the OECD is without p

4、rejudice to the status of the Golan Heights,East Jerusalem and Israeli settlements inthe West Bank under the terms of international law.Note by the Republic of TrkiyeThe information in this document with reference to“Cyprus”relates to the southern part of the Island.There is no singleauthority repre

5、senting both Turkish and Greek Cypriot people on the Island.Trkiye recognises the Turkish Republic ofNorthern Cyprus(TRNC).Until a lasting and equitable solution is found within the context of the United Nations,Trkiyeshall preserve its position concerning the“Cyprus issue”.Note by all the European

6、Union Member States of the OECD and the European UnionThe Republic of Cyprus is recognised by all members of the United Nations with the exception of Trkiye.Theinformation in this document relates to the area under the effective control of the Government of the Republic of Cyprus.Please cite this pu

7、blication as:OECD(2024),Finnish Education Evaluation Centre(FINEEC):OECD Centre for Skills Evaluations,OECD Skills Studies,OECD Publishing,Paris,https:/doi.org/10.1787/b1c0b194-en.ISBN 978-92-64-78522-9(print)ISBN 978-92-64-80407-4(PDF)ISBN 978-92-64-66301-5(HTML)ISBN 978-92-64-98560-5(epub)OECD Ski

8、lls StudiesISSN 2307-8723(print)ISSN 2307-8731(online)Photo credits:Cover Prostock-studio/S.Corrigenda to OECD publications may be found at:https:/www.oecd.org/en/publications/support/corrigenda.html.OECD 2024 Attribution 4.0 International(CC BY 4.0)This work is made available under the Creative Com

9、mons Attribution 4.0 International licence.By using this work,you accept to be bound by the terms of this licence(https:/creativecommons.org/licenses/by/4.0/).Attribution you must cite the work.Translations you must cite the original work,identify changes to the original and add the following text:I

10、n the event of any discrepancy between the original work and the translation,only the text of original work should be considered valid.Adaptations you must cite the original work and add the following text:This is an adaptation of an original work by the OECD.The opinions expressed and arguments emp

11、loyed in this adaptation should not be reported as representing the official views of the OECD or of its Member countries.Third-party material the licence does not apply to third-party material in the work.If using such material,you are responsible for obtaining permission from the third party and f

12、or any claims of infringement.You must not use the OECD logo,visual identity or cover image without express permission or suggest the OECD endorses your use of the work.Any dispute arising under this licence shall be settled by arbitration in accordance with the Permanent Court of Arbitration(PCA)Ar

13、bitration Rules 2012.The seat of arbitration shall be Paris(France).The number of arbitrators shall be one.3 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Foreword The Finnish Education Evaluation Centre(FINEEC)plays a crucial role in monitoring and assessing Finlands education system.Establi

14、shed in 2014,FINEEC has earned a reputation as an impartial and credible institution,providing invaluable data and insights that support educational institutions and policy makers.As Finland faces evolving challenges such as declining performance in international assessments,unmet employment outcome

15、s for vocational education and training(VET)graduates,and the integration of new technological tools,it is essential to examine how FINEEC can adapt and enhance its contributions to policy making.This report,Finnish Education Evaluation Centre(FINEEC):OECD Centre for Skills Evaluations,presents a co

16、mprehensive analysis of FINEECs scope,quality,and usefulness,focusing on areas for improvement.Through self-assessments,in-person visits,workshops with international experts,and extensive research,the evaluation identifies key findings and suggests pathways for strategic enhancement.The report highl

17、ights the necessity for FINEEC to clarify its role in policy making,ensure flexibility in its Evaluation Plan,and balance its ambitious undertakings with available resources.It also emphasises the importance of producing causal evidence to support policy decisions,leveraging data to understand the i

18、mpact of institutional policies on student outcomes,and improving learning assessments in VET.While the focus of this study is on identifying areas for improvement,it is important to acknowledge FINEECs significant contributions to the Finnish education system.FINEECs enhancement-led evaluation appr

19、oach has been instrumental in fostering a culture of quality within schools,higher education institutions,and municipalities.Its thematic reports provide valuable insights into educational trends and its assessments of learning outcomes help institutions benchmark and improve their performance.The r

20、elevance of this study extends beyond Finland.As a benchmark in education,Finlands experiences offer valuable lessons for other countries facing similar challenges.By sharing its approaches and strategies,Finland contributes to the global dialogue on education,fostering collaborative learning and co

21、ntinuous improvement.The OECD hopes that this report serves as a valuable resource for policy makers,educators and stakeholders,providing insights and practical tools to strengthen the governance and quality of education systems globally.The OECD can support countries in their efforts to improve edu

22、cation systems through comprehensive evaluations.The OECD Centre for Skills Evaluations provides rigorous and independent analysis that leads to meaningful insights and recommendations,helping align educational practices with evolving societal and economic needs.4 FINNISH EDUCATION EVALUATION CENTRE

23、(FINEEC)OECD 2024 Acknowledgements The OECD wishes to thank the staff members of the Finnish Education Evaluation Centre(FINEEC)and members of the FINEEC Board for their indispensable contributions to this evaluation.Their engagement in interviews,focus groups and bilateral meetings has been vital t

24、o the projects success.Special thanks to FINEEC Director Harri Peltoniemi;Hannele Seppl,Vice Director of FINEEC;and Senior Manager Mari Rkklinen for their openness and co-operation.Their comprehensive responses to the self-assessment questionnaire,prompt answers to queries and substantial support in

25、 mission organisation have been critical to the success of this evaluation.We are also very grateful to the Ministry of Education and Culture of Finland staff,especially Aleksi Kalenius,Ministerial Advisor at the Ministry of Education and Culture,for their guidance,unwavering support and crucial rol

26、e in mission organisation.We sincerely appreciate various stakeholders whose insights have greatly enriched our understanding.This includes representatives from the Finnish National Agency for Education,the Association of Finnish Municipalities,municipalities,universities,and vocational education an

27、d training(VET)institutions.Their contributions have been invaluable.We extend our thanks to the international experts who assisted with this project,including Helen Arus(Estonia),Head of General Education of the Estonian Quality Agency for Education(HAKA);Enis Dogan,Chief Psychometrician of the US

28、National Centre for Education Statistics(NCES);and Kim Eun Young,Director of Office of International Cooperation of the Korean Educational Development Institute(KEDI).We would also like to thank OECD experts Diana Toledo Figueroa,Project Leader of the OECD Education Policy Outlook,and Simon Roy,Seni

29、or Policy Analyst,Higher Education Policy and Benchmarking,and others,who participated in our workshops.Their shared experiences,ideas and reflections on Finlands challenges and potential solutions have been instrumental.Additionally,we are grateful for the participation of Jukka Mattila,senior mini

30、sterial adviser of the Ministry of Finance;Niku Mttnen,chairperson of The Finnish Economic Policy Council;Sirkku Kupiainen,visiting researcher of the Centre for Educational Assessment CEA at the University of Helsinki;and Roope Uusitalo,professor at the University of Helsinki,who joined the launch e

31、vent to discuss the results of the project.Their participation provided a rich opportunity to have a rich conversation on how the current education evaluation system can contribute more strongly to the improvement of public policies.Ricardo Espinoza and Patricio Ruedi were the OECD project leaders r

32、esponsible for co-ordinating this evaluation and were the principal authors of this report.Special mention must be made of our OECD colleagues,particularly Andrew Bell,Head of OECD Skills Strategies,and Marieke Vandeweyer,Team Leader for Vocational Educational and Training,who provided analytical gu

33、idance and supervised the project.El Iza Mohamedou,Head of the OECD Centre for Skills,oversaw the project,while Stefano Scarpetta,OECD Director for Employment,Labour and Social Affairs,and Mark Pearson,OECD Deputy Director for Employment,Labour and Social Affairs,provided strategic oversight.Finally

34、,we extend our gratitude to Bastiaan de Laat and Mauricio Torres Velasquez from the 5 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 OECDs Evaluation and Internal Audit Unit and to Carlos Hinojosa of the Public Governance Directorate for their invaluable methodological guidance during the earl

35、y stages of this project.Mikaela Malka,an intern for the OECD Centre for Skills,supported the co-ordination of missions and contributed to literature research and information analysis.Darragh Mckee Mathews,also an intern at the OECD Centre for Skills,assisted in the editing and publication process o

36、f the report.Finally,we acknowledge Vronique Qunehen,Duniya Dedeyn and Tara Byrne for their excellent work in mission organisation and Jennifer Cannon for her communication support.6 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Abbreviations and acronyms The following are the main acronyms c

37、ited in this report:Abbreviation/acronym Full description CEDEFOP European Centre for the Development of Vocational Training DAC Development Assistance Committee ECEC Early Childhood Education and Care EDUFI Finnish National Agency for Education ENQA European Association for Quality Assurance in Hig

38、her Education EQA External Quality Assurance ESG Standards and Guidelines for Quality Assurance in the European Higher Education Area ESU European Students Union FINEEC Finnish Education Evaluation Centre FINHEEC Finnish Higher Education Evaluation Council HE Higher Education HEI Higher Education In

39、stitution OECD Organisation for Economic Co-operation and Development OKM Ministry of Education and Culture(Finland)PDCA Plan,Do,Check,Act PISA Programme for International Student Assessment QA Quality Assurance UAS University of Applied Sciences 7 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 202

40、4 Table of contents Foreword 3 Acknowledgements 4 Abbreviations and acronyms 6 Executive summary 9 1 Overview 11 Introduction 12 Educational context in Finland 12 Structure of the Finnish education system 13 The Finnish Education Evaluation Centre(FINEEC)14 References 22 2 Methodology 24 Objective o

41、f the evaluation 25 Evaluation framework 25 Evaluation methodology 26 Reference 29 3 Evaluation results 30 Overall perception of FINEEC 31 FINEECs strategy and scope of work 31 FINEECs operations 39 FINEECs contribution to policy making 43 References 54 4 Conclusions 58 Focus and scope of the evalua

42、tion 59 Summary of key findings 60 Recommendations for moving forward 61 Annex A.Theory of Change 63 Reference 66 8 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Annex B.Evaluation matrix 67 Annex C.Institutions met during the mission to Finland in March 2023 70 Annex D.Evaluation activities

43、and focus areas defined in the Evaluation Plan 2020-2023 71 Reference 71 Annex E.The evaluation setup for the evaluation of learning outcomes in VET,as defined in the Evaluation Plan 2020-2023 72 Reference 72 Annex F.Review of a sample of FINEECs reports 73 FIGURES Figure 1.1.Trends in performance o

44、f 15-year-olds in reading,mathematics and science,Finland 13 Figure 1.2.The quality system of FINEEC 19 Figure 1.3.FINEECs organisational structure 20 Figure 1.4.FINEECs stakeholders and users of evaluation information 21 Figure 2.1.Dimensions and sub-dimensions selected for the study 26 Figure 3.1

45、Employment of graduates one year after graduation in Finland,2011-2020 50 Figure A A.1.Theory of change 65 Figure A A.2.Theory of change:Project assumptions and risks 66 Figure A D.1.Evaluation activities 71 Figure A E.1.The evaluation setup for the evaluations of learning outcomes in VET 72 TABLES

46、Table 1.1.Enhancement-led evaluation principles 15 Table 1.2.Types of evaluations conducted by FINEEC across different education levels 16 Table 4.1.Key findings regarding FINEECs strategy and scope of work 60 Table 4.2.Key findings regarding FINEECs operations 60 Table 4.3.Key findings regarding FI

47、NEECs contribution to policy making 60 Table A B.1.Evaluation matrix 67 Table A F.1.FINEECs reports reviewed 74 9 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Executive summary This study presents a detailed examination of the Finnish Education Evaluation Centre(FINEEC),focusing on the scope

48、,quality and usefulness of its activities within the Finnish education system,and specifically highlights areas for improvement.FINEEC was established on 1 May 2014 by merging the evaluation functions of the Finnish National Agency for Education(EDUFI),the Education Evaluation Council,and the Finnis

49、h Higher Education Evaluation Council(FINHEEC)pertaining to university assessments.It oversees the evaluation of the entire education system and is engaged in both domestic and international evaluation initiatives.FINEEC is highly regarded both nationally and internationally as a credible and releva

50、nt institution.This reputation is acknowledged by various actors in the education sector,as well as by prestigious international institutions,including the European Association for Quality Assurance in Higher Education(ENQA).FINEECs mandate encompasses a wide range of educational levels and forms,pr

51、imarily focusing on conducting external assessments using an enhancement-led evaluation approach.This method emphasises continuous improvement by collaborating with educational institutions and policy makers to develop the Finnish education system through rigorous evaluation,data analysis and eviden

52、ce-based decision making.This study revolves around three areas:the scope of FINEECs evaluation activities;the quality of these evaluations;and their utility from a policy perspective.However,it is important to note that this study does not directly assess FINEECs performance in all areas.The evalua

53、tion concentrates solely on FINEECs activities to the extent that they are related to or influence FINEECs capacity to contribute to policy making.Furthermore,the evaluation highlights only the identified areas for improvement.By concentrating exclusively on areas for improvement,many areas where FI

54、NEEC excels are not discussed in detail.Indeed,it should be noted that FINEEC enjoys a high reputation among users of its work domestically and among its international peers.This study utilises the OECDs Development Assistance Committee(DAC)Evaluation,from which relevance,coherence and effectiveness

55、 were selected as key criteria.The three criteria correspond directly to the dimension selected for evaluation:“relevance”evaluates the extent to which FINEECs activities address the necessary and desirable tasks within the educational sector(scope),“coherence”examines the robustness of FINEECs meth

56、ods and their alignment with international standards(quality);and“effectiveness”assesses the practical impact of FINEECs outputs on education policy and development(usefulness).This evaluation employs various methods:a self-assessment by FINEEC staff;in-person visits to Finland to collect data,inclu

57、ding via interviews and focus groups;two workshops with international experts;and comprehensive desktop research.These methods aim to provide an in-depth understanding of FINEECs work,its strengths,and,particularly,its areas for improvement.The report identifies nine key findings highlighting areas

58、for improvement within FINEECs operations and methodologies.The report does not prescribe specific recommendations(which is outside the scope of this evaluation study);instead,it highlights the main issues and challenges currently faced by FINEEC and 10 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OEC

59、D 2024 provides some illustrative examples of actions that could be taken to remedy them.The report does not highlight the many areas where FINEEC performs very well;it concentrates solely on areas for improvement.By presenting these alongside international evidence and examples,the report aims to i

60、nspire policy reform and strategic enhancement in FINEECs approach to evaluation.Key findings FINEECs strategy and scope of work 1.There is a lack of agreement on the role FINEEC should play in policy making.The exact role FINEEC should undertake in proposing or developing solutions based on its rep

61、orts is unclear,leading to ambiguity among stakeholders.2.There is disagreement about FINEECs autonomy in defining the Evaluation Plan.Views vary on the scope of FINEECs autonomy,impacting collaboration with other institutions in the development of the Evaluation Plan.3.FINEECs Evaluation Plan could

62、 provide greater flexibility to adapt to evolving priorities.The Evaluation Plan is considered insufficiently flexible,restricting FINEECs ability to respond to emerging educational challenges and priorities.FINEECs operations 4.FINEECs undertakings sometimes exceed its resources,which could negativ

63、ely impact the quality of its output.Balancing an ambitious Evaluation Plan against available resources presents challenges,potentially affecting the quality of FINEECs work.5.Seeking greater complementarity between FINEECs lines of work could help generate new insights.FINEEC has the opportunity to

64、 enhance integration among its various evaluations,leading to deeper,more comprehensive insights.FINEECs contribution to policy making 6.Unequal access to results of learning outcomes assessments may undermine education equality.The access model to FINEECs assessments may create disparities,particul

65、arly disadvantaging smaller or less wealthy municipalities.7.A greater focus on producing causal evidence would substantially contribute to policy making.The Finnish education evaluation system needs to develop more causal evidence to guide effective policy making.The role of FINEEC in this respect

66、needs to be clarified.8.Data could be better leveraged to improve understanding of the relationship between institutional-level policies and student outcomes.FINEEC could use data more effectively to reveal the connections between educational policies and student performance.9.Strengthening learning

67、 assessments in VET could support more informed policy making.FINEEC faces challenges in accurately assessing VET learning outcomes due to the reliance on non-standardised teacher grades,highlighting a need for more reliable assessment methods.11 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024

68、This chapter provides an overview of Finlands education system and the role of the Finnish Education Evaluation Centre(FINEEC)within it.It details FINEECs evaluation activities,which include assessing learning outcomes,conducting thematic and system evaluations,performing quality management assessme

69、nts and carrying out audits.The chapter also outlines FINEECs objectives,stakeholders and contributions to educational improvement and policy development.Additionally,it addresses various ongoing and emerging challenges that prompted the decision to evaluate FINEEC.1 Overview 12 FINNISH EDUCATION EV

70、ALUATION CENTRE(FINEEC)OECD 2024 Introduction Finland has long been recognised globally for its strong education system,emphasising student-centric learning,individual growth,equality and quality teaching.Finlands approach,which makes limited use of standardised testing and is characterised by a sig

71、nificant degree of teacher autonomy,has fostered an environment where students can excel both academically and personally.The equality and accessibility of the Finnish education system have supported strong academic achievement at the national level in the past two decades,as evidenced by its high r

72、anking in the OECDs Programme for International Student Assessment(PISA).As a result,Finland has received significant global recognition for its strong education system and achievements.However,while Finlands PISA performance has been very strong in international comparison,it has declined steadily

73、over the past two decades,raising concerns and questions about the factors contributing to this negative trend.In 2014,Finland established the Finnish Education Evaluation Centre(FINEEC)to oversee the evaluation of all aspects of the countrys education system,with a goal of continuous improvement.FI

74、NEECs tasks range from evaluating early childhood education to adult learning.Since its inception,the Finnish Education Evaluation Centre(FINEEC)has played an important role within the Finnish education sector.In addition to conducting evaluations,FINEEC generates both information and recommendation

75、s for the local,regional and national levels to support decision making and development activities as well as international comparisons.It is widely recognised as an impartial,credible and relevant institution,both nationally and internationally,with its outputs serving as invaluable tools for educa

76、tional institutions.For instance,FINEEC is an active member of the European Association for Quality Assurance in Higher Education(ENQA)and the recent ENQA assessment of FINEEC found it to be a well-respected and mature agency,highlighting its strong enhancement-oriented mission and effective stakeho

77、lder engagement within the Finnish higher education system(ENQA Agency,20221).The OECD Centre for Skills has evaluated FINEEC with the aim to identify areas for improvement in order to enhance its capacity to respond to Finlands emerging educational challenges and to strengthen its role in promoting

78、 the development of the Finnish education system.The focus of this project is to identify areas for improvement;therefore,this report does not describe in detail areas where FINEEC excels.This study utilises the OECDs Development Assistance Committee(DAC)Evaluation Criteria(OECD,20212).Using methodo

79、logies such as self-assessment,Theory of Change,stakeholder interviews,workshops and document analysis,the OECD provides a thorough and independent assessment of FINEEC.However,it is important to note that this evaluation does not intend to exhaustively examine all facets and areas of FINEECs operat

80、ions.Instead,it aims to provide an in-depth understanding of FINEECs work,its strengths,and,particularly,its areas for improvement with respect to the scope,quality and usefulness of FINEECs activities.Educational context in Finland During the past decades,Finnish students have achieved remarkable l

81、earning outcomes,showcasing their competence in critical thinking,problem solving,and overall academic performance.By making minimal use of standardised tests,emphasising quality teaching and strongly emphasising teacher autonomy,Finland has cultivated an educational system that empowers students to

82、 flourish academically.These factors,combined with a focus on equal access to education for all students,have solidified Finlands position in educational excellence on the global stage.Finlands education system has consistently performed above average in international comparisons,notably in the OECD

83、s PISA test.However,since 2006,Finland has been witnessing a gradual absolute decline in learning outcomes and performance in PISA,causing concerns about the future of its education system(OECD,20223)(see Figure 1.1).This underscores the necessity for collecting evidence on the activities and perfor

84、mance of 13 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 the Finnish education system through evaluations and,in turn,for examining the role and output of FINEEC.The decline in Finlands PISA scores has outpaced international trends,prompting an interest in better understanding its causes.The

85、 Ministry of Education and Culture(OKM)has identified potential causes,including reductions in educational funding and growing socio-economic disparities in the aftermath of the recession of the 1990s(Kalenius,20234).Other contributing factors have been identified,including a declining number of sch

86、ools in certain regions,reduced funding for basic education and an increasing number of children from financially challenged backgrounds lacking adequate home learning support.Understanding these factors and their implications for the education system is crucial for policy makers and stakeholders to

87、 devise effective strategies to maintain Finlands position as a global leader in education.Figure 1.1.Trends in performance of 15-year-olds in reading,mathematics and science,Finland Note:White dots indicate mean-performance estimates that are not statistically significantly above/below PISA 2022 es

88、timates.Black lines indicate the best-fitting trend.An interactive version of this figure is available at https:/oecdch.art/a40de1dbaf/C096.Also see www.oecd.org/publication/pisa-2022-results/country-notes/finland-6991e849/f.Source:OECD(20223),PISA 2022 Database,Tables I.B1.5.4,I.B1.5.5 and I.B1.5.6

89、,www.oecd.org/pisa/data/2022database/.Structure of the Finnish education system Finlands education system consists of several levels,beginning with early childhood education and care(ECEC),which is provided for children before compulsory education begins.Pre-primary education is provided by local au

90、thorities for children in the year preceding the start of compulsory education.Primary and lower secondary education,delivered by comprehensive schools,is followed by upper secondary education,offered in the form of either general upper secondary education or vocational education and training(VET).H

91、igher education is provided by universities and universities of applied sciences(Kalenius,20234).See OECD(20245)for a visualisation of the Finnish education system.Adult education is also available and offers a wide range of alternatives from primary to higher education.Compulsory education starts i

92、n the year children turn seven.After completing nine years of primary and lower education,individuals can choose between general upper secondary education,vocational upper secondary education and training,or a double degree that consists of vocational qualification and a matriculation examination.Ge

93、neral upper secondary education leads to a matriculation examination diploma,while vocational education leads to a vocational qualification.In Finland,the education system offers a high degree of flexibility,allowing individuals to switch paths and pursue further studies at any level.Furthermore,the

94、 Finnish education system ensures equal access to educational opportunities for everyone,regardless of background or circumstances.Notably,in Finland,education is entirely free,starting from preschool and extending to higher education,and teachers are held 14 FINNISH EDUCATION EVALUATION CENTRE(FINE

95、EC)OECD 2024 in high regard(OECD,20206).At the same time,education providers enjoy a high degree of autonomy compared to their peers internationally.With respect to the governance of the Finnish education system,the Ministry of Education and Culture(OKM)is responsible for drafting education-related

96、legislation and implementing the government programme.EDUFI is the national development agency responsible for developing the core curricula and qualification requirements for ECEC,pre-primary,basic,general and vocational upper secondary education,as well as for adult education and training.For thei

97、r part,local municipalities manage certain practical aspects of education,such as creating the municipal curriculum,which steers instruction and schoolwork in more detail,taking local needs and perspectives into consideration.The Finnish education system stands out for its decentralised approach and

98、 lack of strong top-down quality control mechanisms.The evaluation system is mainly based on national sample tests(especially in primary and lower secondary education),as well as the measurement and promotion of quality management(especially in higher education and VET).Education institutions are le

99、gally obligated to assess the quality and impact of the education they deliver and engage in external reviews of their operations.This responsibility is outlined in the specific laws governing each level of education,from ECEC to tertiary education.The methods for conducting the self-assessments are

100、 determined at the local level.Municipal schools are supervised by local school boards that are also responsible for the self-evaluations.The Finnish Education Evaluation Centre(FINEEC)Objectives and activities FINEEC was established on 1 May 2014 by merging the evaluation functions of EDUFI,the Edu

101、cation Evaluation Council,and the Finnish Higher Education Evaluation Council(FINHEEC),which was responsible for undertaking university assessments.FINEECs jurisdiction encompasses the entire education system,including ECEC,preschool and basic education,upper secondary education,vocational education

102、,free educational work,basic art education,adult education,and higher education.In addition to its domestic role,FINEEC actively engages in international evaluation initiatives and collaborates with other organisations in the field,such as the International Network for Quality Assurance Agencies in

103、Higher Education.FINEECs primary goal is to conduct assessments using an enhancement-led evaluation approach.This method fosters continuous improvement by collaborating with educational institutions and policy makers.Table 1.1 presents the four principles that guide this approach.15 FINNISH EDUCATIO

104、N EVALUATION CENTRE(FINEEC)OECD 2024 Table 1.1.Enhancement-led evaluation principles Principles Description Goal orientation and futures consciousness FINEECs evaluations focus on current practices,foresee development trends,and factors fostering or hindering progress.They aid continuous improvement

105、 and consider both present and future aspects.Participatory and interactive approach FINEEC encourages open discussions and trust among participants,utilising evaluation data for mutual benefit.Transparency is upheld by sharing criteria and conclusions,fostering collaboration and utilising existing

106、resources.Tailored methods FINEEC adapts evaluation methods to suit each case,producing comprehensive data from diverse angles.Methods evolve based on insights during the evaluation process.Supporting changes FINEEC promotes learning,participation,and awareness,offering feedback on strengths,areas n

107、eeding development,and best practices.Outcomes link to wider frameworks,and impact is tracked through feedback and follow-ups.Source:Authors elaboration based on official information from FINEEC,(20207),Enhancement-led Evaluation at the Finnish Education Evaluation Center,www.karvi.fi/en/publication

108、s/enhancement-led-evaluation-finnish-education-evaluation-center.Overall,FINEECs mandate is to promote the development and improvement of the Finnish education system through rigorous evaluation,data analysis and evidence-based decision making.FINEEC generates both information and recommendations fo

109、r the local,regional and national levels to be used in decision making and development activities as well as international comparisons.FINEEC conducts five types of national evaluations:1.Evaluations of learning outcomes:These evaluations produce national-level information on the achievement of the

110、objectives of the curricula and students competencies.FINEEC analyses and reports the results and communicates the main findings to the relevant stakeholders.FINEEC conducts evaluations of learning outcomes in basic education and secondary education.The purpose of these evaluations is to enhance tra

111、ining,facilitate learning and ensure the quality of education.The evaluations provide valuable information about education and training at the local,regional and national levels.This information aids in decision making,supports development initiatives and enables international comparisons.2.Thematic

112、 evaluations:These evaluations produce comprehensive information about the evaluation subject and give developmental recommendations.Thematic evaluations frequently explore whether changes in education policy have achieved the intended outcomes.Some recommendations target education providers and edu

113、cational institutions;others target the OKM and EDUFI.3.System evaluation:These evaluations assess the strengths and areas for improvement within a specific education level or the entire education system.Additionally,they assess the effectiveness of initiatives aimed at reforming and enhancing the o

114、verall education system.4.Quality management assessments:Another of FINEECs roles is to assess how well education providers(at all levels)manage their tasks and resources to deliver quality education.FINEEC also supports education providers in using evaluation information,improving their evaluation

115、capacities and using effective evaluation methods.5.Audits:These aim to assess the quality of education provided by higher education institutions,which usually involves evaluating curriculum design,teaching methodologies,assessment strategies and the overall learning environment.From 2020 to 2022,FI

116、NEEC published a total of more than 200 reports and organised 305 events,seminars and conferences,which were attended by more than 14 000 participants.Additionally,a significant number of education and training providers,as well as educational institutions and higher education institutions,engaged i

117、n FINEECs evaluations,with a combined total of more than 11 000 participants.16 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Table 1.2 shows the different types of evaluations FINEEC undertakes at each educational level.Table 1.2.Types of evaluations conducted by FINEEC across different educ

118、ation levels Education level FINEECs legal mandate(as stipulated in the legislation)Evaluation of learning outcomes Thematic and system evaluation reports Quality systems,audits and evaluations Transversal to all levels of education Evaluate education and education providers in accordance with an Ev

119、aluation Plan.X X X Support education providers in matters related to evaluation and quality management.Develop the evaluation of education.Attend to any other duties that are issued or given to FINEEC.Early childhood education and care Carry out evaluations related to ECEC and the activities of ECE

120、C,as well as teaching and training providers,in accordance with the Evaluation Plan.X Thematic evaluations in early childhood education focus on a content-based entity or subject area.In system evaluations,the whole of the education system or its sub-area can also be examined.The subject of the eval

121、uation can be education policy and its implementation or the processes of reforming and developing the education system.Between 2020 and 2023,FINEEC developed the national digital quality evaluation system,Valssi.The system provides tools and processes to evaluate the structural and process-related

122、factors of ECEC.It produces information about the quality of ECEC on a national,municipal/private provider and ECEC provider.The system is designed to support the quality management and continuous development of ECEC providers and strengthen the involvement of all important stakeholders in the evalu

123、ation process.Support early childhood education,teaching and training providers in matters concerning evaluation and quality management.Develop evaluation of education and early childhood education.The early childhood education quality assessment system implements early childhood education evaluatio

124、ns and collects and produces evaluation information to support early childhood education development.Pre-primary,primary,secondary and general upper secondary education Undertake evaluations of learning outcomes relating to the distribution of lesson hours and the national core curriculum targets re

125、ferred to in Section 14 of the Basic Education Act(628/1998)and in Section 10 of the General Upper Secondary Schools Act(629/1998).The purpose of the evaluations of learning outcomes is to produce reliable information about the achievement of the objectives of the basic curricula of pre-primary and

126、basic education,the level of competence in the subject and the realisation of equality in education(i.e.by relating the results to students background Thematic evaluations of pre-primary and basic education focus on a content package or theme.The subject can also be an evaluation of the state of a c

127、ertain form of education.In system evaluations,the whole of the education system or some part of it can also be examined.The subject of the evaluation can be In the area of quality management,FINEECs task is to support education providers in matters related to quality management and evaluation.17 FI

128、NNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Education level FINEECs legal mandate(as stipulated in the legislation)Evaluation of learning outcomes Thematic and system evaluation reports Quality systems,audits and evaluations factors and the features of the learning environment).education poli

129、cy and its implementation or the processes of reforming and developing the education system.Evaluations of learning outcomes relating to the national core curriculum targets referred to in Section 5 of the Act on Basic Education in the Arts(633/1998).The assessment of learning outcomes in basic educ

130、ation is based on direct evaluation of outcomes by large-scale surveys.Vocational education and training Evaluate education and education providers in accordance with an Evaluation Plan referred to in Section 5 of the Act on Basic Education in the Arts(633/1998).The learning outcome evaluation aims

131、to produce information on how well the students have achieved the learning objectives and vocational competence requirements defined in the qualification requirements.In addition,the learning outcome evaluations look at the quality of the pedagogical activities of the training providers.Theme and sy

132、stem evaluations produce information to support decision making and educational development.Theme evaluations focus on a content-based entity or subject area.In system evaluations,the object of the evaluation is the education system or its sub-area.In the area of quality management,FINEECs task is t

133、o produce information on the state of the quality management systems of vocational training providers(evaluations of quality management systems)and to support training providers in matters related to quality management and evaluation.Undertake,in accordance with the Evaluation Plan referred to in Se

134、ction 5 of the Act on Basic Education in the Arts(633/1998),evaluations of learning outcomes relating to targets of the national core curriculum referred to in Section 13 of the Vocational Education and Training Act(630/1998)and in the national qualification requirements referred to in Section 13 of

135、 the Vocational Adult Education Act(631/1998).In VET,learning outcomes are assessed indirectly by the grades the institutions award the students.Support education providers in matters related to evaluation and quality management.Higher education Carry out evaluations related to education,education p

136、roviders and the activities of higher education institutions in accordance with the Evaluation Plan.Theme evaluations focus on a particular theme within the education system.System evaluations centre on the entire education system or part The developmental evaluation in higher education follows the

137、European standards and guidelines for quality assurance(ESG 2015),and the aim is to involve the higher education 18 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Education level FINEECs legal mandate(as stipulated in the legislation)Evaluation of learning outcomes Thematic and system evaluati

138、on reports Quality systems,audits and evaluations Support education providers and higher education institutions in evaluation and quality management matters.of it and can cover the evaluation of education policies/reforms and their implementation.institutions personnel,students and stakeholders in i

139、dentifying the strengths,good practices and development areas of the higher education institutions operations.The goal is also to support higher education institutions in achieving their own goals and thus create conditions for the continuous development of higher education institutions.Note:FINEECs

140、 responsibilities and structure are defined in the Act on the National Education Evaluation Centre(1295/2013)and the corresponding regulation(1317/2013).These legal frameworks outline the tasks and organisational framework within which FINEEC operates and define its independence.Source:FINEEC(20248)

141、,National Education Evaluation Center,https:/karvi.fi/;Ministry of Education and Culture,Finland(20139),Act on the Finnish Education Evaluation Centre,www.finlex.fi/fi/laki/ajantasa/2013/20131295.According to the information shared by FINEEC through a self-assessment questionnaire,the Centre uses a

142、quality management model to guide its operations.This model is based on Demings continuous improvement cycle,known as PDCA(Plan,Do,Study,Act),which involves a cycle of planning,implementation,monitoring and evaluation and aims to generate continuous learning and improvement within the organisation(s

143、ee Figure 1.2)(The Deming Institute,202410).This supports the strategy,steering,and management and enhances FINEECs operational responsiveness to needs and priorities.19 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Figure 1.2.The quality system of FINEEC Source:FINEEC(202111),Self-Assessment

144、 Questionnaire,www.karvi.fi/sites/default/files/sites/default/files/documents/KARVI_10_2021_FINAL.pdf.Operations and budget FINEEC is an independent authority defined by the Act on the National Education Evaluation Centre(1295/2013).It operates as a separate unit within EDUFI under the OKM,as specif

145、ied in Section 564 of the Act on the Finnish National Agency for Education(2016/6).In 2023,FINEEC employed 53 employees and a network of around 200 external experts.About 40%of the employees were employed on temporary contracts.The annual costs associated with its operations amounted to approximatel

146、y EUR 4.8 million.These expenses were covered by the annual appropriation for operational expenditure,amounting to EUR 3 122 000 in 2022 and EUR 4 409 000 in 2023.Of this budget,over EUR 1 million was committed directly to the new digital system for assessing learning outcomes in basic education(Jod

147、a)and the digital quality assessment system in ECEC(Valssi).FINEEC also receives funds from paid services,as defined in the Ministry of Education and Culture Decree 160/2021,Section 3.In addition,FINEEC receives funding from the OKM to carry out specific studies and several long-term evaluations.Thi

148、s funding covers only the incremental costs,which include salaries and personnel costs of temporary experts and other costs arising from the projects;it does not cover the remuneration of FINEECs permanent staff.20 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Structure and governance FINEEC

149、operates as a distinct unit within EDUFI and is independent in its operations.EDUFI provides FINEEC with some administrative and support services,such as human resources,financial,legal,procurement and data management services.FINEEC comprises the Evaluation Council,the Higher Education Evaluation C

150、ommittee and four units:the General Education;the Vocational Education and Early Childhood Education Unit;the Higher Education and Liberal Adult Education Unit;and the Development Services Unit(see Figure 1.3).The main office and most of the staff are in Helsinki;some staff are located in Jyvskyl(EN

151、QA Agency,20221).Figure 1.3.FINEECs organisational structure Source:FINEEC(n.d.12),FINEECs organisation,www.karvi.fi/en/about-us/fineecs-organisation.FINEEC operates under the authority of the Evaluation Council,which consists of 13 members,and the Higher Education Evaluation Committee,which compris

152、es 9 members.The Government Decree on FINEEC(1317/2013)prescribes the Councils composition,tasks and decision-making powers.The Council plays an essential role in the monitoring and strategic orientation of FINEEC.They actively participate in strategic planning,make decisions on significant statemen

153、ts and initiatives,contribute to preparing the evaluation plans,and monitor their implementation.Their involvement ensures the alignment of assessment processes with organisational objectives and drives progress in evaluating educational practices.The Council prepares a proposal for an evaluation pl

154、an in accordance with Section 1295 of the Act on the National Education Evaluation Centre(2013/5)and amendments to the evaluation plan,which the OKM approves.Council members are representatives from different educational sectors,teacher education,research,working life and students.The Council select

155、s the chair and vice-chair from among its members.After consulting with the Council,the Director of FINEEC formally approves all project plans for evaluations and compositions of planning and evaluation teams for all education sectors apart from higher education.21 FINNISH EDUCATION EVALUATION CENTR

156、E(FINEEC)OECD 2024 The Higher Education Evaluation Committee is the decision-making body for higher education evaluations and consists of nine members,of which three must also be members of the Council.According to the Government Decree,the Council proposes members for the Evaluation Committee,and t

157、he OKM appoints them.The Council selects candidates from the candidates put forward by higher education institutions and other stakeholders.The members must be experts in the evaluation of higher education.The Evaluation Committee also selects the chair and vice-chair from among its members.The Eval

158、uation Committee decides on project plans and compositions of planning and evaluation teams for evaluations of higher education institutions and the outcomes of audits of higher education institutions(ENQA Agency,20221).FINEECs stakeholders and users of evaluation information Figure 1.4 shows FINEEC

159、s stakeholders and users of evaluation information,categorised by their level of priority.The highest priority category encompasses the Evaluation Council and the Higher Education Evaluation Committee,as well as staff and providers of education and early childhood education and care(ECEC),as well as

160、 pupils.The second highest priority category includes the media,the Ministry of Education and Culture(OKM),the Finnish National Agency for Education(EDUFI),alternating decision makers,and external evaluation and specialist groups.The third priority category consists of all other actors,such as citiz

161、ens,other ministries and other stakeholders.Figure 1.4.FINEECs stakeholders and users of evaluation information Source:FINEEC(202313),Self-assessment questionnaire for OECD External evaluation of FINEEC.The context in which FINEEC operates FINEEC functions within Finlands multifaceted network of edu

162、cational evaluation and assessment entities.Its contributions are crucial and form part of a larger,collective effort involving various organisations and institutions.This collaborative framework is designed to provide a comprehensive understanding of the Finnish educational landscape.22 FINNISH EDU

163、CATION EVALUATION CENTRE(FINEEC)OECD 2024 International assessments such as the OECDs PISA,as well as studies conducted by the International Association for the Evaluation of Educational Achievement(IEA),such as Progress in International Reading Literacy Study(PIRLS)and Trends in International Mathe

164、matics and Science Study(TIMSS),are managed outside of FINEEC.These activities are co-ordinated directly by the OKM,with implementation support from prominent educational research bodies,including the Finnish Institute for Educational Research at the University of Jyvskyl,which has been evaluating l

165、earning outcomes since the 1960s,and the Centre for Educational Assessment at the University of Helsinki,which has been active since the 1990s.FINEECs work in mathematics encompasses conducting national assessments that,alongside international studies like PISA,create a comprehensive view of educati

166、onal outcomes.FINEECs national data provides valuable insights into mathematics learning outcomes,while international assessments contribute broader trend analysis across various subjects,including literacy.In the domain of thematic evaluations,FINEEC functions amid a diverse array of organisations

167、that evaluate various aspects of the Finnish education system,including specific policy reforms.For instance,qualitative evaluations of higher education and continuous learning systems have been reviewed by international bodies such as the OECD.Similarly,consultancy groups and other evaluative bodie

168、s have examined the impact of university reforms and legislation changes on vocational and general upper secondary education during the coronavirus(COVID-19)pandemic and the implementation of the VET reform and ECEC data collection.Econometric impact assessments,which often require specialised analy

169、tical skills,are predominantly carried out by research universities and economics research institutes.For example,the VATT(Valtion Taloudellinen Tutkimuskeskus)Institute for Economic Research and other institutions have evaluated the effects of reforms in higher education admissions systems with fun

170、ding from the OKM.Furthermore,a collaborative approach to evaluation is evident in joint analysis,assessment,and research activities(VN TEAS),which are co-ordinated by the government and involve institutions like VATT,the University of Helsinki and Aalto University.These entities are currently evalu

171、ating a randomised controlled trial(RCT)on extending pre-primary education,showcasing Finlands comprehensive and collaborative nature of educational research.References ENQA Agency(2022),Review of the Finnish Education Evaluation Centre(FINEEC),https:/www.enqa.eu/wp-content/uploads/FINEEC-external-r

172、eview-report.pdf.1 FINEEC(2024),National Education Evaluation Center,https:/karvi.fi/.8 FINEEC(2023),Self-assessment questionnaire for OECD External evaluation of FINEEC.13 FINEEC(2021),Self-assessment Report for the ENQA Agency Review 2021,https:/www.karvi.fi/sites/default/files/sites/default/files

173、/documents/KARVI_10_2021_FINAL.pdf.11 FINEEC(2020),Enhancement-led Evaluation at the Finnish Education Evaluation Center,https:/www.karvi.fi/en/publications/enhancement-led-evaluation-finnish-education-evaluation-center.7 23 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 FINEEC(n.d.),FINEECs O

174、rganisation,https:/www.karvi.fi/en/about-us/fineecs-organisation(accessed on 29 April 2024).12 Kalenius,A.(2023),Sivistyskatsaus 2023,Opetus-ja kulttuuriministeri,https:/julkaisut.valtioneuvosto.fi/handle/10024/164564.4 Ministry of Education and Culture,Finland(2013),“Act on the Finnish Education Ev

175、aluation Centre”,https:/www.finlex.fi/fi/laki/kaannokset/2013/en20131295.pdf(accessed on 1 February 2024).9 OECD(2024),OECD Education GPS,https:/gpseducation.oecd.org/CountryProfile?primaryCountry=FIN.5 OECD(2022),PISA 2022 Database,Tables I.B1.5.4,I.B1.5.5 and I.B1.5.6,https:/www.oecd.org/pisa/data

176、/2022database/.3 OECD(2021),Applying Evaluation Criteria Thoughtfully,OECD Publishing,Paris,https:/doi.org/10.1787/543e84ed-en.2 OECD(2020),TALIS 2018 Results(Volume II):Teachers and School Leaders as Valued Professionals,TALIS,OECD Publishing,Paris,https:/doi.org/10.1787/19cf08df-en.6 The Deming In

177、stitute(2024),PDSA Cycle,https:/deming.org/explore/pdsa/(accessed on 1 February 2024).10 24 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 This chapter outlines the objectives and methodology employed to assess the work of the Finnish Education Evaluation Centre(FINEEC).The chapter begins by p

178、resenting the objectives of the evaluation,which included assessing the scope,quality and usefulness of FINEECs activities.It then presents the evaluation framework and the dimensions evaluated,including the relevance,coherence and effectiveness of FINEECs work.The chapter also provides a detailed d

179、escription of the evaluation methodology,which involves desk research,self-assessments,expert panels,interviews,focus groups and theory of change.2 Methodology 25 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Objective of the evaluation The objective of this study is to assess the scope of th

180、e Finnish Education Evaluation Centres(FINEECs)work and its quality,considering whether FINEEC is carrying out the tasks that are most needed and/or desirable;the quality of the analysis is sound;and the results are useful.More specifically,this study:Assesses the scope of FINEECs evaluation activit

181、ies:The analysis looks at the subjects covered in FINEECs evaluation activities and the distribution of its activities over the different parts of the education system to identify gaps and imbalances.Assesses the quality of FINEECs evaluations:This analysis looks at whether the tools and methods use

182、d in FINEECs evaluations are suitable to fulfil its mandate and in line with international practices.Assesses the usefulness of FINEECs activities from a policy perspective at the system level and for education institutions:This analysis evaluates whether the distribution of FINEECs activities is ap

183、propriate from a policy perspective.It also assesses whether the outcomes from FINEECs evaluation activities form a reliable base for making policy decisions,provide well-grounded and implementable policy recommendations,and contribute effectively to Finlands education policy priorities.Evaluation f

184、ramework This study employs the OECDs Development Assistance Committee(DAC)Evaluation Criteria(OECD,20211).These evaluation criteria provide an internationally recognised framework and a structured approach for evaluating FINEECs activities.The criteria are widely recognised as playing a central rol

185、e in improving the quality of global evaluation practice and supporting collaboration.They have enabled organisations to design and deliver evaluations that are relevant to the needs of decision makers and capture a wide range of intended and unintended results,producing valuable evidence and insigh

186、ts.This study concentrates on three evaluation criteria:relevance,coherence and effectiveness These criteria were selected to align with the objectives of the external evaluation of FINEEC mentioned above and after discussion with the Finnish Ministry of Education and Culture(OKM)on the study.Two gu

187、iding principles accompany the criteria.First,“the criteria should be applied thoughtfully to support high-quality,useful evaluation”.Second,“use of the criteria depends on the purpose of the evaluation”.In other words,the criteria should be applied thoughtfully and adapted to the context of the int

188、ervention and the evaluation.The selected criteria allow for the examination of whether FINEEC is carrying out the most needed and/or desirable tasks and whether the quality of FINEECs analysis is sound and producing results useful for education providers and policy makers.Each criterion,referred to

189、 as“dimensions”for the purposes of this evaluation,is composed of two or three sub-dimensions.The dimensions with their corresponding sub-dimensions are presented in Figure 2.1.The dimension“Relevance”is evaluated by assessing how well FINEECs objectives,strategies and planning align with the needs

190、of beneficiaries,national policies and the priorities of partner institutions.It also allows for an assessment of FINEECs ability to adapt to changing policy environments.This dimension has three sub-dimensions.The first sub-dimension,“Responsiveness to needs,policies and priorities”,assesses the ex

191、tent to which FINEEC addresses stakeholder concerns.The second sub-dimension,“Sensitivity and responsiveness to context”,analyses how FINEECs goals and tasks evolve in response to changing economic,environmental,social,cultural and political circumstances.The third sub-dimension,“Quality of design”,

192、assesses whether FINEECs structure and organisation effectively facilitate the pursuit of relevant goals and whether these goals are clearly defined.26 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Figure 2.1.Dimensions and sub-dimensions selected for the study Source:Authors elaboration base

193、d on OECD(20211),Applying Evaluation Criteria Thoughtfully,https:/doi.org/10.1787/543e84ed-en.The dimension“Coherence”is evaluated by assessing the alignment between FINEECs objectives and tasks both within the organisation and in relation to other actors in the education system.This dimension inclu

194、des two sub-dimensions.The first sub-dimension,“External coherence”,assesses the compatibility of FINEECs objectives and work with other actors interventions in the education sector,nationally and abroad.The second sub-dimension,“Internal coherence”,assesses the interconnectedness and synergies betw

195、een all interventions undertaken by FINEEC.The final dimension,“Effectiveness”,is evaluated by assessing the degree to which FINEEC has achieved its intended objectives and outcomes,considering variations in results across services and beneficiary groups.This dimension has three sub-dimensions.The f

196、irst sub-dimension,“Achievement of objectives”,examines whether FINEECs interventions have successfully produced the desired outcomes.The second sub-dimension,“Symmetry of results across levels of education and local authorities”,evaluates the inclusiveness and fairness of outcomes among diverse ben

197、eficiary groups.The third sub-dimension,“Factors influencing results”,analyses positive and negative effects originating from FINEEC,the OKM,local authorities,or the broader national context that contribute to the attainment or non-attainment of FINEEC objectives.The evaluation of FINEECs performanc

198、e across these dimensions makes it possible to reflect more deeply on FINEECs strengths and challenges with respect to contributing to the continuous improvement of the education system in Finland.Evaluation methodology This evaluation combines different methodologies:desktop research,self-assessmen

199、ts,interviews,focus groups and Theory of Change.It also involved consulting with a wide range of FINEECs stakeholders and organising workshops with international experts to learn from their experiences,thereby facilitating reflection on FINEECs role and performance.Evaluation strategy Once the dimen

200、sions to be examined during the evaluation were defined and validated with the technical counterparts(i.e.the OKM),an evaluation matrix was constructed.The evaluation matrix is a tool that allows for the development of a set of guiding questions based on the dimensions and sub-dimensions.27 FINNISH

201、EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 The matrix also sets out the methodologies to be employed and information sources to be explored in order to answer these questions.Data collection instruments,including a self-assessment questionnaire and interview and focus group guides,were also drafte

202、d.The information sources to be consulted were identified jointly with the studys technical counterparts and with FINEEC to ensure that the main stakeholders were consulted during the evaluation.An abbreviated version of the matrix can be found in Annex B.The list of stakeholders consulted during th

203、e study is as follows:Ministry of Education and Culture(OKM)o Minister and Permanent Secretary o Department for General Upper Secondary Education and Vocational Education and Training(VET)o Department for Higher Education and Science Policy o Department for Early Childhood Education,Comprehensive Sc

204、hool Education and Liberal Adult Education.Other educational administration institutions o Finnish National Agency for Education o Evaluation Council o Higher Education Evaluation Committee.Local authorities and education providers o Association of Finnish Municipalities o Education officials from s

205、ix municipalities o Association of Finnish Education Employers o The Finnish Association for the Development of Vocational Education and Training o Directors of six VET providers.FINEEC o Directors and Strategic Advisors o Shared Development Service Unit o General Education Unit o Vocational Educati

206、on and Early Childhood Education Unit o Higher Education and Liberal Adult Education Unit o Managers for focus areas o Process Managers o Methodology Team.Self-assessment questionnaire A self-assessment questionnaire was designed and shared with FINEEC staff.This self-assessment included more than 3

207、0 questions related to the dimensions and sub-dimensions selected for analysis in this evaluation.The responses provided by FINEEC staff were supported by documentation to support or further elaborate on aspects of their responses.Responses to the self-assessment questionnaire are available from FIN

208、EEC upon request.28 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Missions Two missions to Finland were undertaken to conduct interviews,run focus groups and collect first-hand insights from FINEEC staff and key stakeholders.The responses received from the initial self-assessment questionnair

209、e by FINEEC staff were instrumental in refining the guidelines for the interviews and focus groups planned for these missions.Each mission,spanning four days each,involved comprehensive discussions with FINEECs management and staff and a selection of stakeholders previously identified as critical to

210、 the evaluation process.These interactions allowed for a deeper exploration of the self-assessment questionnaire responses and provided valuable perspectives on FINEECs operations and impact.Workshops with international experts Two workshops were held with the participation of international experts,

211、OECD experts,FINEEC staff,and stakeholders to facilitate reflection on FINEECs activities and performance and the range of possible courses of action that Finland could take.They convened international experts,including specialists from the National Center for Education Statistics of the United Stat

212、es,the Korean Educational Development Institute,and the Estonian Quality Agency for Education.In terms of OECD experts,specialists in Education Policy and Higher Education Policy and Benchmarking participated.All these experts were selected due to their extensive experience in the evaluation of educ

213、ation systems as well as to illuminate the diversity of possible evaluation approaches.In these workshops,some of the findings of the initial consultation process carried out by the OECD with FINEEC and its stakeholders were presented to highlight some of the challenges FINEEC faces in actively cont

214、ributing to improving the Finnish education system.For their part,international experts presented information about the organisation and the role of the institutions they represented,as well as the assessment models used in their respective countries.This permitted reflection on points of similarity

215、 or difference with regard to the work carried out by FINEEC.These presentations inspired further discussions on the challenges faced by FINEEC and how these could be addressed in the future.Desktop research Several internal documents and research literature were consulted throughout the evaluation

216、with three main objectives in mind.The first objective was to identify and choose possible evaluation frameworks and methodologies relevant to a rigorous and high-quality evaluation of FINEEC.The second was to examine documentation and reports produced by FINEEC in order to better understand its mis

217、sion,organisational structure,activities and outcomes.The final objective was to analyse literature on the current performance of the Finnish education system that would provide context for this evaluation.A guide was prepared for the review of the reports published by FINEEC.Then,with the help of a

218、rtificial intelligence,a random selection of FINEEC reports was analysed(see Annex F for details).This analysis covered four key areas:objectives,methodology,conclusions and quality of recommendations.In relation to the objectives,the purpose,type and scope of the evaluation were analysed.In terms o

219、f methodology,the types of data and statistical approaches used in the evaluations were analysed,distinguishing between quantitative and qualitative analyses.Regarding the conclusions,the analysis looked at whether the evaluation findings support these and whether the limitations of the conclusions(

220、and the data supporting them)are addressed in a transparent manner within the reports.Finally,the analysis of the quality of recommendations examined the specificity,feasibility and empirical basis of the recommendations given by FINEEC in its reports and whether recommendations are made at a genera

221、l level or the level of each stakeholder.Annex F presents the guidance and the results of the analysis of the reports.29 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Limitations of the methodology It is important to acknowledge the limitations of the methodology used in this study.Although c

222、omprehensive research was conducted,including interviews with FINEEC staff,the OKM and stakeholders;an international literature review;a review of reports produced by FINEEC;and discussions with global experts in workshops,these approaches have limitations.First,interviews and stakeholder responses

223、may be subject to personal or institutional biases.Second,the literature review might not encompass all current or emerging perspectives.Third,the workshop discussions rely on the participants knowledge and experiences,which might not fully capture the diversity of perspectives across all relevant s

224、ectors.Fourth,the evaluation method involved reviewing only a random sample of FINEEC reports.Reference OECD(2021),Applying Evaluation Criteria Thoughtfully,OECD Publishing,Paris,https:/doi.org/10.1787/543e84ed-en.1 30 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 This chapter presents the re

225、sults of the evaluation of the Finnish Education Evaluation Centre(FINEEC).It begins with an overview of perceptions of FINEEC,highlighting its reputation as an impartial and credible institution.The chapter then identifies nine key areas for improvement,organised around the themes of FINEECs strate

226、gy and scope of work,operations and contributions to policy making.For each of the challenges identified,strategies and actions for moving forward are highlighted.3 Evaluation results 31 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Overall perception of FINEEC Based on the information gather

227、ed during the interviews with stakeholders in Finland,the Finnish Education Evaluation Centre(FINEEC)is seen as an impartial,relevant and credible institution that provides regular information on the status of the Finnish education system through the evaluations and assessments it conducts.This asse

228、ssment is supported by findings from the latest FINEEC stakeholder survey conducted in 2022,which received responses from 312 participants and achieved a 23%response rate.The survey aimed to assess FINEECs success in evaluation activities,usage of evaluations,its reputation among stakeholders,the ef

229、fectiveness of stakeholder co-operation and communication,and considerations for the upcoming evaluation plan.Results indicated that 89%of respondents consider FINEECs reputation to be at least“fairly good”,reflecting a 9%improvement compared to the same measurement in 2018 and a 5%.improvement to 2

230、020.Its positive reputation is largely attributable to FINEECs acknowledged expertise,strong stakeholder engagement practices and perceived impartiality.These three elements could be considered part of the main added value of FINEEC.Stakeholders positive assessments of FINEECs expertise are based on

231、 its longstanding experience in conducting evidence-based evaluations and national assessments.Positive perceptions of its stakeholder engagement performance are attributed to its employment of enhancement-led evaluation methods,which involve consulting with stakeholders at various stages of the eva

232、luation process,promoting awareness of the strengths and weaknesses of the organisation(or target)being evaluated,and supporting change in the organisation.Lastly,FINEECs perceived impartiality has been founded largely on its operational and financial autonomy,which protects it from potential third-

233、party interests.Notwithstanding the generally positive perceptions about the work and attributes of FINEEC,there remains room for improvement.It is important to note that this evaluation aims to identify potential areas for improvement within FINEECs scope of evaluations,as well as their quality and

234、 relevance to policy making,rather than examining every aspect of its operations.For this reason,the findings presented below do not cover the numerous areas where FINEEC excels and for which it is internationally recognised.The remainder of this report presents nine key areas for improvement,organi

235、sed into three broad categories to provide a structured overview.These categories are FINEECs strategy and scope of work,FINEECs operations,and FINEECs contribution to policy making.Additionally,international examples are included to show how other countries have addressed similar challenges and to

236、serve as inspiration for future policy reforms.However,these examples are not intended to be interpreted as recommendations;they are provided to underscore the possibility for improvement and stimulate thinking of enhancements in FINEECs approach to evaluation.FINEECs strategy and scope of work Key

237、Finding 1.There is a lack of agreement on the role FINEEC should play in policy making While FINEECs evaluative role is recognised and valued by many stakeholders,there is a lack of consensus on the role FINEEC should play in proposing or developing solutions to the problems identified in its report

238、s.This disagreement centres around whether FINEEC should actively propose policy recommendations or concentrate on providing the data and analysis to facilitate a broader dialogue on possible solutions.The mandate of FINEEC does not explicitly require it to provide recommendations,which leaves its r

239、ole open to interpretation This lack of agreement on FINEECs role in the policy-making 32 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 process creates ambiguity that can detract from the use of its core evaluation findings.There is also disagreement about what constitutes sufficiently robust

240、 evidence to underpin its recommendations,a topic that will be explored in depth in other sections(see Key Finding 7).Evidence There is a divergence in opinions about FINEECs role in proposing specific policy solutions to the problems illuminated by its evaluations.While FINEEC staff believe they ar

241、e responsible for making recommendations and contributing to developing solutions to problems in the education system,its stakeholders generally do not expect FINEEC to fulfil this role.This disagreement is rooted in two main factors.First,FINEECs recommendations are often perceived as overly broad

242、and not well grounded in evidence(see Key Finding 7),and questions have been raised about their practical applicability.Second,the prevailing view among stakeholders is that the Finnish National Agency for Education(EDUFI)should be the primary entity responsible for proposing solutions,with FINEECs

243、role being more concentrated on conducting evaluations and presenting information and analysis to stimulate discussion about potential solutions.For example,education providers argue that FINEECs primary role should be to assess the education system,with the aim of generating quality information on

244、students trends performance,as well as on the education systems strengths and challenges.FINEEC,according to the results of the latest stakeholder survey,fulfils this role quite well:91%report that FINEEC produces information on current and relevant issues through the evaluations it conducts,and 82%

245、report that FINEEC evaluations contribute to the development of the Finnish education system.In addition,education providers recognise FINEECs role in informing public discussion on issues associated with Finlands education system,a role that,from their perspective,could be strengthened in the futur

246、e.This latter perspective is supported by the FINEEC stakeholder survey results,with only 57%of respondents reporting that they“somewhat agree”or“completely agree”with the statement“FINEEC participates actively in the societal debate on education”.Furthermore,education providers noted that FINEEC sh

247、ould leverage its reputation,expertise and the trust it has garnered to promote and inform evidence-based dialogue and decisions.One of the approaches FINEEC already employs to do this is issuing statements,which have mostly been made in response to parliamentary requests see,for example(FINEEC,2023

248、1;FINEEC,20232;FINEEC,20233).FINEECs increased engagement in public discussion would not only strengthen the quality of policy dialogue but could also lead to more effective and evidence-based educational policies,ultimately enhancing the Finnish education system.From EDUFIs perspective,FINEECs adde

249、d value lies in its impartiality as well as its competence and experience in conducting evaluations on strategic and/or long-term issues,where the assessment of learning outcomes plays a fundamental role.For this reason,in EDUFIs view,FINEEC could prioritise its limited resources on these activities

250、.For its part,the OKM has noted that the methodological approaches used by FINEEC do not provide a sufficient basis for developing concrete policy recommendations,primarily because the methods and analysis undertaken by FINEEC typically do not produce causal evidence(Key Finding 7.).Furthermore,the

251、OKM holds that FINEEC is not expected to deliver policy recommendations but that if it decides to do so,these recommendations must be based on causal evidence,which is not always currently the case.Like the OKM,education providers and EDUFI representatives did not see FINEEC as the organisation resp

252、onsible for generating policy recommendations or proposing solutions to the problems identified in their evaluations.33 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 A review of a sample of FINEECs reports by the OECD corroborates the perception that policy recommendations are not sufficientl

253、y grounded in evidence(see Annex F).For example,the report Equality and Participation in Education-An Overview of National Evaluations,published in 2021(FINEEC,20214),makes several recommendations for dismantling inequality-producing structures and ensuring continuous learning opportunities for all

254、socio-economic backgrounds.While these recommendations are broadly consistent with the need for equality in education identified by FINEEC,they are not directly supported by the findings of the evaluation or by external literature.Further evidence can also be found in the set of recommendations pres

255、ented by FINEEC in the 2022 report,Partnering with Working Life-Evaluation of Workplace Education and Training and Working Life Co-operation in Vocational Education and Training(FINEEC,20225).These recommendations emphasise improving the effectiveness of workplace education and promoting closer co-o

256、peration between vocational education and training(VET)providers and employers.While the recommendations align with the reports overall conclusions,highlighting the need to better integrate VET with working life,they lack support from specific empirical data or external research.Similarly,in a 2022

257、report on vocational qualifications in electrical engineering and automation technology(FINEEC,20226),FINEEC presented recommendations to improve vocational education practices.The report advocated for increased participation of workplace representatives in assessments and a dedicated workplace inst

258、ructor for each student,enhanced teacher-instructor collaboration,specialised training for instructors,and teacher work placements.These proposals appear to be driven by an understanding of educational needs,but limited empirical support or detailed analysis is provided for these specific recommenda

259、tions.While FINEEC has made recent efforts to make the recommendations more concrete and targeted,for example,by making distinct recommendations for each implicated party see,for example,FINEEC(20225),most stakeholders consulted still considered its recommendations inadequate as a basis for setting

260、policy.Some education providers argued that it may be hard for FINEEC to generate useful recommendations for education providers as their contexts and needs are typically unique;therefore,it is challenging to generate recommendations that are applicable and useful to all.Among the stakeholders consu

261、lted,there was a broad consensus about the roles of EDUFI and the OKM.Education providers interviewed see EDUFI as a crucial partner in proposing solutions and directly supporting schools in implementing educational programmes.This perspective on EDUFIs role is consistent with its own vision for its

262、elf.For its part,stakeholders see the OKM as being responsible for developing legislation and providing resources for the functioning and improvement of the education system.Moving forward Clarity about the role of FINEEC in policy making will be important to ensure that scarce resources are used ef

263、fectively and efficiently.There is a need,therefore,for key actors in the education sector to engage in a frank discussion about FINEECs role in this regard and reach a formal consensus.One possible way forward is for FINEEC to concentrate its resources on its evaluation role.By limiting its role in

264、 developing policy recommendations,FINEEC could allocate more resources to enhancing its existing evaluation practices.This strategic shift would not only free up resources to devote to evaluations themselves but would also allow more resources to be devoted to disseminating the organisations findin

265、gs,thereby strengthening its contribution to the public discourse and fostering a broader consensus on educational evaluation policies(see Box 3.1 to see how providing access to evidence can contribute to building consensus on educational evaluation policies).For example,this could be achieved throu

266、gh more frequent publications of statements and ensuring that all FINEECs publications receive wider 34 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 dissemination(see Box 3.6 later in this chapter;it includes an example of an effective dissemination strategy from Switzerland).Box 3.1.Using e

267、vidence to inform consensus building A significant challenge for policy makers lies in the transition from identifying necessary changes to successfully implementing them.Implementing educational evaluation policies is a complex process involving diverse stakeholders with distinct interests,necessit

268、ating informed debates and capacity building.The resistance to reform could stem from stakeholders inadequate information about proposed policy changes,their impact and the potential individual or group outcomes.This lack of awareness may be exacerbated by the unpreparedness of public opinion for ce

269、rtain reforms,leading to a lack of social acceptance for innovative policies,especially in the context of an underdeveloped culture and limited tradition of evaluation in education.It underscores the need to promote research and make the evidence supporting policy proposals accessible to relevant st

270、akeholders.The goal is to raise awareness of issues,foster a national debate and disseminate evidence on the effectiveness and impact of various policy alternatives,ultimately aiming to build a consensus on educational evaluation policies.Source:OECD(20137),Synergies for Better Learning:An Internati

271、onal Perspective on Evaluation and Assessment,https:/doi.org/10.1787/9789264190658-en.Having FINEEC specialise more narrowly in the conduct of evaluations is consistent with the practices of a number of other countries,including the United States,where research and evaluation tasks in the education

272、sector are allocated to distinct entities,which may or may not be under the umbrella of a single organisation(Box 3.2 presents the role of the various public centres focused on advancing education research in the United States).This division or roles allows each entity to focus on a narrower set of

273、functions,facilitating the development of specific competencies and experience that,other things being equal,can translate into better quality.When these entities are grouped under the responsibility of the same organisation,collaboration can be facilitated,and resources can be used more efficiently

274、.This example illustrates possible ways of dividing the roles related to evaluation and research within the education sector among different stakeholders and underscores the importance of these stakeholders working together and complementing each others efforts.Box 3.2.International example:The stru

275、cture of the US Institute of Education Sciences The US Institute of Education Sciences comprises four centres and ten Regional Educational Laboratories,each with distinct roles in advancing education research in the United States:National Center for Education Research(NCER):NCER supports research to

276、 address significant education problems,aiming to enhance education quality,boost academic achievement,reduce gaps and improve access to post-secondary education.NCERs research spans diverse areas,including reading and writing,mathematics and science education,teacher quality and leadership.Its task

277、s include identifying impactful programmes and policies,developing new interventions(e.g.curricula,teacher professional development programmes),evaluating the efficacy and effectiveness of specific interventions,and developing and validating evaluations,thereby contributing to the understanding of t

278、eaching and learning.National Center for Education Statistics(NCES):As the primary statistical agency,NCES collects,compiles and analyses comprehensive statistical information on American education.35 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 National Center for Education Evaluation and R

279、egional Assistance(NCEE):Specialising in impact studies,NCEE evaluates federally funded education programmes,particularly in reading,mathematics and science,providing crucial insights for education policy and practice.National Center for Special Education Research(NCSER):NCSER advances practices for

280、 teaching children with disabilities through research grants,supporting interventions and validating measures.Regional Educational Laboratories(RELs):The Institute of Education Sciences operates ten RELs,which collaborate with educators and policy makers to generate and apply evidence,aiming to enha

281、nce learner outcomes.Their work is intended to be change-oriented,rigorous and impactful,supporting decisions on education policies,programmes,and practices at local,regional or statewide levels.RELs contribute research to understand variations in educational experiences based on context and student

282、 groups,influencing outcomes and identifying potential solutions.This strategic distribution of tasks among the four centres ensures a comprehensive and focused approach to education research.While NCER and NCEE address broader educational challenges and programme evaluations,NCES concentrates on st

283、atistical data collection,and NCSER specifically targets research related to special education.In addition,RELs enhance the use of the evidence generated to propose specific and targeted solutions to schools in need.This collaborative and divisional structure enhances the efficiency and effectivenes

284、s of the Institutes efforts to improve education outcomes in the United States.Source:IES(n.d.8),Institute of Education Sciences,https:/ies.ed.gov/.Narrowing the scope of FINEECs work could allow it to spend more time explaining the findings of its evaluations in detail.The evaluations conducted by

285、FINEEC provide crucial insights for EDUFI,education providers,and the OKM,thereby informing their decision-making processes.However,ensuring that FINEECs findings are communicated effectively is essential to enhance policy formulation.Establishing robust communication channels between FINEEC,the OKM

286、,and EDUFI is critical to facilitate this.Implementing formal mechanisms for regular discussions among these entities about FINEECs evaluation outcomes could lead to more targeted and effective policy responses to identified issues.These discussions need not always culminate in immediate policy acti

287、ons but could instead highlight areas requiring further investigation or alternative solution testing.In this regard,each organisation could play to its strengths:EDUFI could lead in designing innovative solutions,the OKM could allocate resources for piloting and assessing these solutions,and FINEEC

288、 could oversee the evaluation process.This approach is not unprecedented;for example,in 2020,FINEEC evaluated a pilot project on free early childhood education for five-year-olds,assessing its impact on participation rates,costs and organisational aspects(FINEEC,20209).Such evaluations provide the O

289、KM with critical data,informing more grounded and effective policy decisions.Key Finding 2.There is disagreement about FINEECs autonomy in defining the Evaluation Plan Although FINEECs autonomy is legally defined and widely recognised as crucial for safeguarding the integrity of its work,there is so

290、me divergence in how stakeholders interpret its scope and its practical implications.This discrepancy is primarily observed between the OKM and FINEEC itself.Many of the stakeholders interviewed,including FINEEC,interpret the autonomy of FINEEC not only to mean that it has independence in terms of h

291、ow it conducts its evaluations and publishes its findings but also that it is 36 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 subject to minimal steering by the OKM in relation to the evaluations selected to be part of the Evaluation Plan.While some stakeholders consider this independence vi

292、tal for ensuring FINEECs objectivity and the credibility of its findings,others express concern that it reduces FINEECs incentives to adapt to changing priorities,particularly those of the OKM.This disagreement about the nature of FINEECs autonomy leads to substantial challenges,notably with regard

293、to the nature of communication between the OKM and FINEEC.For example,the OKM has expressed concern that any requests or proposals it may make with respect to the work of FINEEC could be regarded as a directive and be perceived as infringing on FINEECs autonomy.This concern is further compounded by

294、the fact that the OKM is a key funder of FINEEC.For these reasons,the OKM may be less than fully clear in articulating its needs to FINEEC.Evidence Interviews with and surveys of FINEECs stakeholders reveal that FINEECs autonomy is widely recognised as being important for safeguarding the integrity

295、of its work from undue external interference.FINEECs autonomy is established in its governing legal framework and is reinforced by its financial independence.The act establishing FINEEC states,“The Finnish Education Evaluation Centre operates in the capacity of an independent expert organisation for

296、 external evaluations of education”(Ministry of Education and Culture,Finland,201310).This autonomy is widely understood to grant it the freedom to independently decide how it conducts evaluations,including its choice of methods and evaluation team members,timetables,scope of reports and other decis

297、ions pivotal to educational evaluations.Autonomy also protects FINEEC from third-party influence,including the OKM,EDUFI and education providers.However,there are some notable differences in how this autonomy is interpreted by FINEEC on the one hand and the OKM on the other.These differences in unde

298、rstanding have practical implications,particularly in undermining effective communication between the two entities and,thus,also undermining the timely measurement of the impact of reforms being implemented.FINEEC tends to interpret its autonomy in a broader,less restrictive manner.Specifically,it i

299、nterprets its autonomy as having the freedom to make independent decisions on various aspects of educational evaluation,including,for example,what topics to evaluate and how,as well as which assessments to conduct at different levels of education.In this interpretation of its autonomy,FINEEC has wid

300、e-ranging operational flexibility.For its part,the OKM views FINEECs autonomy as more limited.The OKM interprets FINEECs autonomy as meaning it has the liberty to choose how it operates,such as the freedom to select its methodologies,tools and internal organisation,but not the right to make independ

301、ent choices about what areas or levels of education to evaluate.Differences in how FINEECs autonomy is interpreted particularly surface in relation to the elaboration of the Evaluation Plan.Although there are several discussions between FINEEC and the OKM prior to the approval of the plan,in practic

302、e,the OKM has little influence over what is included in FINEECs Evaluation Plan(see Key Finding 3).The OKMs lack of influence over what is included in the Evaluation Plan often results in evaluations that lack direct policy relevance for the OKM.From the OKMs perspective,this means it does not have

303、access to the evidence base it needs to support its decision making.To ensure it has the evidence base it needs,the OKM has supplemented FINEECs resources to undertake additional evaluations that the OKM considered strategic but were not included in the original Evaluation Plan.Ministry officials no

304、te,however,that the current period of fiscal restraint will make it more difficult to find supplementary funds to undertake additional evaluations in response to evolving priorities.This funding arrangement has generated some disagreement between FINEEC and the OKM in relation to which evaluations s

305、hould be included in the Evaluation Plan and which should be left out and financed 37 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 through the OKMs development funds.The question of which evaluations should be encompassed or excluded from the Evaluation Plan is further complicated by the abs

306、ence of synchronisation between its development and electoral cycles.As FINEEC,OKM staff and various stakeholders highlighted,this lack of alignment can result in a misalignment between the topics in the Evaluation Plan and the governments current priorities This misalignment could emerge when incom

307、ing governments propose new reforms for evaluation or express the need to assess existing institutions or programmes.Given that the timeline for evaluations may span across multiple electoral cycles,some degree of misalignment is inevitable.The difficulty in adapting the Evaluation Plan to new needs

308、 is developed in more detail in Key Finding 3.Moving forward The disparity in the interpretation of FINEECs autonomy,from the broader,strategic autonomy envisioned by FINEEC and the more operational autonomy envisioned by the OKM,leads to tensions between the organisations and hinders effective comm

309、unication.There is a need to clarify the nature of FINEECs autonomy and the OKMs role in guiding the work of FINEEC.As a first step,FINEEC and the OKM should openly discuss their perspectives and map points of divergence.This step would involve discussing decision-making processes,methodological fre

310、edom and the extent of independence in setting objectives and tasks.Additionally,the two entities could establish regular and structured communication channels,particularly at the strategic level.These channels could facilitate ongoing dialogue,ensuring alignment and efficiently addressing misunders

311、tandings.The dialogue between FINEEC and the OKM,as well as the resolutions taken as a result,could be made publicly transparent.This would protect FINEECs valued and well-regarded autonomy.Key Finding 3.FINEECs Evaluation Plan could provide greater flexibility to adapt to evolving priorities While

312、the process for developing FINEECs four-year Evaluation Plan is often praised for being participatory and deliberative,many stakeholders,including the OKM,report that the resulting plan could be designed with greater flexibility to accommodate undertaking new evaluation topics in line with new or em

313、erging priorities.Although a review and adjustment of the plan is conducted two years after its implementation,the need for greater adaptability remains evident due to concerns from educational institutions over the delays in FINEECs results publication.There is a risk that what was initially deemed

314、 relevant at the start of the cycle may no longer hold the same relevance upon publication.This highlights the importance of improving the ability to identify change or the emergence of new evaluation needs,and to improve the flexibility of the Evaluation Plan to respond to these changes,in order to

315、 maintain the relevance and impact of FINEECs work.Evidence Despite the scheduled review and adjustment of the Evaluation Plan in the second year of its four-year duration(see Box 3.3 for more information on the Evaluation Plan),integrating new evaluation proposals proposed by the OKM remains challe

316、nging in practice.This is partly due to the absence of formalised channels for communicating the governments changing priorities and adapting FINEECs activities accordingly.Furthermore,undertaking new,additional evaluations would entail replacing previously prioritised and planned evaluations.While

317、some argue that this situation helps protect the Evaluation Plan and FINEECs work from political interference and short-term thinking,there is a need to consider how 38 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 new priorities can be accommodated within the lifecycle of the plan without un

318、dermining the organisations autonomy(see Key Finding 3 for further analysis of FINEECs autonomy).Box 3.3.The FINEEC Evaluation Plan(institutional information)The FINEEC Evaluation Plan is a crucial document developed over four years to guide educational evaluations.It is crafted through a consultati

319、ve process led by the Evaluation Council,appointed by the OKM,and operating in conjunction with FINEEC.The process adheres to legislative requirements for broad stakeholder collaboration,ensuring that the plan is co-created with input from key users of the evaluations,including on proposals for futu

320、re themes.This collaboration guarantees that the themes are timely,relevant and resonate with stakeholder needs and motivations.The OKM is responsible for approving the Evaluation Plan.The plans content is informed by a desire to fortify the national knowledge base in education and to integrate find

321、ings from prior research and external sources.It is designed to be responsive and adaptable,aligning with the dynamic nature of educational policies and societal needs.A routine review of the plan is conducted at least once midway(after the second year)through the four-year cycle.This review serves

322、as a reflective exercise to examine the outcomes of completed evaluations and to deliberate on any necessary updates to the plan.For instance,during the coronavirus(COVID-19)pandemic,the plan was promptly adjusted to evaluate the impact of the crisis on learning,demonstrating its capacity for flexib

323、ility in the face of unforeseen circumstances.As in the original Evaluation Plan,the OKM is responsible for approving any subsequent modifications based on the Councils proposals.Despite this adaptability,the four-year span of the plan also offers stability,providing predictability for education pro

324、viders and ensuring that evaluation results can be seamlessly integrated into ongoing development projects.This stability is also strategic in preventing overlap with other surveys and assessments.Source:FINEEC(202411),Plan for Education Evaluation 20202023,www.karvi.fi/en/about-us/about-our-evaluat

325、ions/plan-education-evaluation-2020-2023.Like the OKM,education providers have expressed similar concerns about the lack of responsiveness of the Evaluation Plan.While educational institutions widely recognise and value FINEECs work and relevance,viewing it as an impartial entity that produces work

326、of high quality and relevance,concerns have been raised regarding the timing of the publication of FINEECs results.Given the four-year cycle of the plan,some evaluations are not published until the end of this period or later.For example,in the 2020-2023 Plan defined in 2019,the evaluations“Overall

327、evaluation of the qualification system of VET”,“Evaluation of the distance learning pilot for Smi languages”,and“Extending compulsory education:Evaluation of the new forms of guidance counselling”,were planned to be conducted in 2022 or 2023,and therefore the results were published between three and

328、 five years after the need to conduct them was detected.As institutional priorities can shift over these four years and new priorities emerge,some results,by the time of publication,might become less relevant to the institutions.Education providers also pointed out that they are not consulted during

329、 the review and adjustment of the four-year Evaluation Plan on the changing needs for evaluation(carried out two years after its implementation).It is important to note that current legislation requires FINEEC to consult key stakeholders in preparing the plan.Therefore,any changes to the plan should

330、 consider stakeholder consultation processes,be discussed in the Evaluation Council,and be validated by the OKM.39 FINNISH EDUCATION EVALUATION CENTRE(FINEEC)OECD 2024 Moving forward FINEEC could build greater flexibility into its approach to developing and managing its Evaluation Plan.A formal comm

331、unication mechanism between FINEEC and the OKM could be established,particularly at the strategic level,to enhance the flexibility of FINEECs Evaluation Plan.For example,at the time of the mid-term review of the Evaluation Plan,the OKM could make a formal request to FINEEC about the inclusion of new

332、 evaluations in the Plan.FINEEC,for its part,could give a formal response on whether or not to include the assessment into the Plan,and the reasons for its decision.This would enable FINEEC to better align its work with changing governmental priorities without compromising its autonomy.Given that cu

333、rrent legislation requires FINEEC to consult key stakeholders in preparing the plan,it is important that changes to the plan are discussed and approved at least by the Evaluation Council.Other consultation mechanisms could be used in this process of reviewing and adjusting the plan in order to analyse possible changes in assessment needs(e.g.due to possible changes in the context)and to inform the