Impact and effects of external evalution on non-higher education schools

This Project intends to contribute for the production and systematization of knowledge concerning External School Evaluation (ESE), through a theoretical analysis of its models and an empirical study of its impact and effects on schools and communities. The project’s conclusion will give rise to the creation of a national interuniversity observatory on ESE, which is of manifest social interest, namely to those responsible for the design and implementation of educational policies. This project is justified as, after the end of the first cycle of ESE, in 2011, «It is important for an independent institution, to conduct a meta-evaluation study of ESE which may identify the degree of attainment of its designed objectives, and main effects achieved, as well as eventual negative effects» [National Council of Education (CNE, 2010)]. There are no national studies on ESE, therefore its impacts and effects are unknown. External evaluation produces consequences, although difficulties regarding the measurement of evaluation of schools are recognized, according to OECD's report. Thus, the financing of a research project aiming to study the impact and effects of ESE on schools and communities is socially valid. A nation-wide meta-evaluation study, this research project is not in itself an evaluation, designed according to widely applied theoretical and methodological frameworks . The problem can be expressed as an interrogation – what impact and effects did the ESE produce during its first cycle of implementation, from 2006/07 to 2010/11, taking as reference school improvement, community participation and the implementation of accountability policies? – leading to the following research questions: Does ESE produce impact and effects on school improvement at the organizational, curricular, and pedagogical levels?; on the consolidation of institutional self-evaluation?; on the appropriation by the school and the community of the referents used for institutional evaluation?(Annex 1). Two general objectives are expressed: to explore theoretical and methodological frameworks on ESE, including its impact and effects on school improvement in the context of accountability policies; to undertake empirical studies centered on the impact and effects of external evaluation. Specific objectives are presented in the Annex 2. This problem justifies the research design and the option for a combination of quantitative and qualitative approaches, according to a mixed model of data gathering which stresses documental analysis, interviews and questionnaires. The research includes an effectiveness evalaution, considering all aspects of an assessment of impact and effects. All the team members, from 6 Portuguese universities, have taken part in evaluation studies and have a deep knowledge of the research object. Researchers have experience teaching supervision at post-graduate courses, having some of them coordinated such courses, which is a decisive factor for the proposal and for the accomplishment of factors related to advanced training. The team’s national dimension is justified by the fact the ESE follows and implementation model divided into 6 regions, according to the inspection’s functional model. The organization of each team includes researchers from universities located at the North, Center, Lisbon and Tejo Valley, Alentejo and Algarve regions. As there is no control group for ESE, the implementation of an efficiency study requires the research to be based on the universe of all schools evaluated, according to a sample stratified by region, as well as the singularity of each school which will be studied by random cases. This project is of manifest social relevance as it responds to the CNE’s (2008) challenge: to pay particular attention to the moment of delivery of the ESE report to the schools, as this is when an important part of the consequences of evaluation for both the schools and the students is at play – to change practices, processes and models according to the ESE’s guidelines. The pertinence of the research project is justified by the increasing social valuing of ESE, and by the non existence of a national study as the CNE (2010) concludes: «there is no systematic study or reflection on the effects of ESE and its usefulness» and, «although we may consider its utility as perceived by those in charge of the evaluated schools to be high, to date it is not possible to identify ESE’s consequences and effects clearly, either at school level, or at the level of those responsible by the design and implementation of policies» Besides making the results public by seminars to be organized at each of the participating Universities, a site will be created, containing bibliographical data, research data, theoretical and methodological frameworks and processes and practices of internal and external evaluation