International surveys about school efficiency and student attitudes
The start is a topic the interest of which goes beyond a single country’s concerns, and about which a comparison between countries or education systems would be enlightening, such as adult understanding level in reading literacy or 18-year-olds’ attitudes towards civism. A group of experts and/or representatives from each country begins to work, most often under the aegis of an international organization (EU, UESCO, OECD, IEA,...). The point is to determine precisely the domains to assess, and to develop a theoretical framework. Then test developers build a test battery. Pilot studies followed by a field trial administered in each participating country help to validate all the test instruments and the administration procedures, in order to found analyses and international comparisons on valid and reliable data. For the main campaign, a population-representative sample is randomly drawn and assessed. Finally, collected data are processed and analyzed.
They help to classify the education systems according to various dimensions – their efficiency, their equity,… (top list).
They enlighten on different realities, other structures, on the fact that our own ways are not universal.
What does seem the rule to us may sometimes be the exception: for instance, in 11 out of the OECD 34 countries, more than 95% students are in the modal class level (which matches our 4th secondary year); in 19 out of 34 countries, the age of the first orientation is 15 or 16 years old.
These surveys help to establish some connections between the characteristics of the systems and their performances, their equity,… Very relevant information can be inferred from this diversity, it is like a giant laboratory. For instance, do the countries where grade retention is applied have better results or is it the opposite? When it was decided in Poland to extend to 1 year the common-core curriculum, did a leveling down occur ?
Of course, comparison has its own limits: ready-made « recipes » cannot be imported without any caution from one country to another. But comparison also provides some tracks and references which are to be taken into account and have more value than mere opinions.
Since its foundation in the early 1960’s by Professor Gilbert de Landsheere, the Laboratoire de pédagogie expérimentale of the University of Liège played a major part in international surveys.
Firstly, at the international level, Gilbert de Landsheere was one of the pioneers who, together with T.N. Postelthwaite and T. Husén, founded the International Association for the Evaluation of educational Achievement, well known as IEA.
In 1964, Belgium took part in a survey about achievement in mathematics, among 12 participating countries. At the end of the 1960’s, French speaking Belgium participated then in Six subjects study. In this time already, the Laboratoire de pédagogie expérimentale was in charge of the implementation of the survey and of the result analysis.
This short summary shows well the founding and dynamic part played by the Laboratoire/Service de pédagogie expérimentale of the University of Liège in the conception and organization of international educational surveys. The current centre aSPe comes definitely within the scope of this foundation.
Many researchers of the University of Liège were and/or are members of international expert committees that monitor international surveys at the highest level. Among the most recent ones, Aletta Grisay played a major part in the PISA development process at the beginning of the OECD-INES A network; for many years, she was a member of the Technical Advisory Group (TAG) of PISA. Christian Monseur is also a member of the TAG of PISA and of I.E.A.-Pirls and he is as well a reference expert in sampling for PISA. In the early 1990’s, Marcel Crahay was a member of the Steering Group of the I.E.A.-PPP (preprimary project). Since the beginning of the PISA programme, Dominique Lafontaine has been a member of the international expert group for reading; she was also a member of such a group for Pirls 2006. For the cycles of PISA 2000, 2009 and 2012, the aSPe was part of the research centres of the consortium that, under the management of the Australian Council of Educational Research (ACER), monitored the PISA surveys. aSPe researchers are also members of the reading items developers (Ariane Baye, Patricia Schillings, and Anne Matoul) and the math items developers (Isabelle Demonty and Annick Fagnant). Some members of the research team were also involved in the French international version translation of the various testing tools, instruments and material resources for PISA 2000, 2009 and 2012. Undoubtedly, the aSPe is a reference centre in the domain of international surveys and its expertise is well-known at the international level.
Finally, the aSPe organizes and monitors the implementation of the international surveys PISA, PIRLS and SurveyLang (European Survey on Languages Competences) at a national level, in the schools of the Wallonia-Brussels Federation, and carries out the analyses of their results.
Baye, A. (2001). Evaluations internationales : Les enjeux du jour et de l'histoire. Les Cahiers du Service de Pédagogie Expérimentale, 7-8, 11-23. Liège, Belgique : Service de Pédagogie expérimentale de l’Université.
http://orbi.ulg.ac.be/handle/2268/84172
Baye, A. (2004). La gestion des spécificités linguistiques et culturelles dans les évaluations internationales de la lecture. Politiques d'Education et de Formation : Analyses et Comparaisons Internationales, 11, 55-70.
http://orbi.ulg.ac.be/handle/2268/84139
Lafontaine, D., & Demeuse, M. (2003). Influence du contenu et des modalités d'évaluation sur les indicateurs d'équités. In Les actes du Colloque de l'Adméé sur le thème Evaluation entre efficacité et équité. Les Cahiers du Service de pédagogie expérimentale, 15-16. Liège, Belgique : Service de Pédagogie expérimentale de l’Université.
http://orbi.ulg.ac.be/handle/2268/80543
Lafontaine, D., & Blondin, C. (2004). Regards sur les acquis des élèves en Communauté française Apports des enquêtes de l’I.E.A, de PISA et des évaluations externes. Bruxelles : De Boeck.
http://orbi.ulg.ac.be/handle/2268/5469
Blondin, C. & Lafontaine, D., (2005). Les acquis scolaires des filles et des garçons en lecture, en mathématiques et en sciences. Un éclairage historique basé sur les enquêtes internationales. Education et francophonie, XXXIII, 1, 37-57. En ligne sur la page http://www.acelf.ca/c/revue/index.php.
Lafontaine, D., & Baye, A. (2007). Au-delà du palmarès : quels enseignements tirer des enquêtes internationales comme PISA ? In CIFoP (Ed.). Nos régions en Europe : voies d’impasse, voies d’avenir. Quel développement pour la Wallonie et Bruxelles. Charleroi : Centre interuniversitaire de formation permanente, 225-247.
http://orbi.ulg.ac.be/handle/2268/28872
Lafontaine, D. (2010). Les standards internationaux en éducation : la place des situations et des contextes in D. Masciotra, F. Medzo & P. Jonnaerts, (Eds). Vers une approche située en éducation. Réflexions, pratiques, recherches et standards, Cahiers scientifiques de l'ACFAS., 55-70.
http://orbi.ulg.ac.be/handle/2268/80534
Lafontaine, D. (2010). Evaluations internationales des systèmes éducatifs et évaluations externes nationales : points de convergences et de tension. Eduquer, 12-16.
http://orbi.ulg.ac.be/handle/2268/74977
Monseur, C., & Lafontaine, D. (2006). Le caractère relatif des indicateurs de tendance. Revue suisse des Sciences de l’Education, 3-28,353-371.
http://orbi.ulg.ac.be/handle/2268/9415
Surveys:
The objective of this international survey, initiated by the 30 countries of the OECD and presently administered in more than 70 countries, consists in measuring the 15-year-old students’ achievements (wherever they are in their school career) in reading, mathematics and science.
A part of the OECD website is entirely devoted to PISA:
The following site also presents a lot of information about PISA in the Brussels-Wallonia Federation:
PISA 2018 Results
Find the results on our website dedicated to PISA 2018 in Wallonia-Brussels Federation :
PISA 2015 Results
PISA explained to everybody: some tracks to improve education in French-speaking Belgium (the video, the présentation)
PISA 2012 Results
15 year-old students faced with computers for PISA 2012, in Federation Wallonia-Brussels
Mathematical culture at the age of 15: PISA 2012 first results in Federation Wallonia-Brussels
Publications
La culture mathématique à 15 ans. Premiers résultats de PISA 2012 en Fédération Wallonie-Bruxelles (2013). Les Cahiers des Sciences de l’éducation, 34. Université de Liège : Service d’analyse des systèmes et des pratiques d’enseignement.
premiers resultats pisa 2012 cahiers 34 .pdf
Les élèves de 15 ans face aux ordinateurs dans le cadre du PISA 2012 en Fédération Wallonie-Bruxelles. Résultats en résolution de problèmes, culture mathématique et lecture sur ordinateur (2014). Les Cahiers des Sciences de l’éducation, 36. Université de Liège : Service d’analyse des systèmes et des pratiques d’enseignement.
face aux ordinateurs cahiers 35 .pdf
Lafontaine, D., & Demeuse, M. (2003). Acquis des élèves à 15 ans et parcours de formation. Capital humain et marchés du travail : perspectives régionales et européennes. Enseignement et formation - marché du travail : quelles articulations ? Actes du 15e Congrès des Economistes belges en langue française, du 28 novembre au 29 novembre 2002. Namur.
http://hdl.handle.net/2268/80541
Monseur, C., Baye, A., & Quittre, V. (2009). Organisation et apports de l'enquête PISA. DIRECT, 16, 45-63.
http://hdl.handle.net/2268/84100
Lafontaine, D., Baye, A, Burton, R., Demonty, I., Matoul, A., & Monseur, C. (2003). Les compétences des jeunes de 15 ans en Communauté française en lecture, en mathématiques et en sciences. Les Cahiers du Service de Pédagogie expérimentale, 13-14. Université de Liège, Belgique : Service de Pédagogie expérimentale.
http://hdl.handle.net/2268/84145
Baye, A, Demonty, I., Fagnant, A., Matoul, A., Monseur, C., & Lafontaine, D., (2006). Les compétences des jeunes de 15 ans en Communauté française en mathématiques, en lecture et en sciences. Résultats de l’enquête PISA 2003. Les Cahiers du Service de Pédagogie expérimentale, 19-20, Service de Pédagogie expérimentale de l’Université.
http://hdl.handle.net/2268/5467
Baye, A., Fagnant, A., Hindryckx, G., Lafontaine, D., Matoul, A., & Quittre, V. (2009). Les compétences des jeunes de 15 ans en Communauté française en sciences, en mathématiques et en lecture. Résultats de l’enquête PISA 2006. Les Cahiers des Sciences de l’Education, 29-30. Université de Liège, Belgique : Unité d’analyse des systèmes et des pratiques d’enseignement.
http://hdl.handle.net/2268/19520
Baye, A., Quittre, V., Hindryckx, G., Fagnant, A., & Lafontaine, D. (2009). Les acquis des élèves en culture scientifique : Premiers résultats de PISA 2006.
http://hdl.handle.net/2268/79743
Baye, A, Demonty, I., Matoul, A., Lafontaine, D., & Monseur, C. (2010). La lecture à 15 ans. Premiers résultats de l’enquête PISA 2009. Les Cahiers des Sciences de l’Education, 31, 1-21. Université de Liège, Belgique : Unité d’analyse des systèmes et des pratiques d’enseignement.
http://hdl.handle.net/2268/79559
Baye, A, Demonty, I., Fagnant, A., Matoul, A., Lafontaine, D., & Monseur, C . (2005). PISA 2003 : au-delà des moyennes, des constats qui forcent à l'action, Les infos de l'Agers - Tables rondes, 1, 2-5.
http://hdl.handle.net/2268/79759
Baye, A, Demonty, I., Fagnant, A., Matoul, A., Monseur, C., & Lafontaine, D., (2004). PISA 2003 : Quels défis pour notre système éducatif ?
http://hdl.handle.net/2268/79760
Lafontaine, D., & Monseur, C. (2009). Gender Gap in Comparative Studies of Reading Comprehension : to what extent do the test characteristics make a difference? European Educational Research Journal. Special issue on PISA and gender. 8(1), 69-79.
http://hdl.handle.net/2268/10908
Reading Litteracy Study for 4th grade students
The objective of this international survey consists in measuring students’ reading comprehension at the 4th year of primary school. It also aims at grasping some elements of the cultural, socioeconomic, education, family and individual background, that can enlighten the achievements observed in each country, as well as the differences of achievement observed between participating countries. That is the reason why PIRLS collects a large range of information with contextual questionnaires administered to the various key players in schools (students, teachers, principals) and to the assessed students’ parents also.
More info
Communication of PIRLS 2016 results
Communication of PIRLS 2011 results
Video capsules
Afin de concrétiser les informations contenues dans la présentation des résultats PIRLS rédigée à l’attention des enseignants du primaire, quatre capsules vidéo ont été réalisées avec le soutien de l'Institut de Formation et de Recherche en Enseignement Supérieur (IFRES) de l’Université de Liège.
Ces capsules, d’une durée qui oscille entre 6 et 12 minutes, abordent successivement quatre thématiques en lien avec l’enseignement/apprentissage de la lecture.
1. Des changements de pratiques professionnelles : des étudiantes du Master en Sciences de l’éducation décrivent la manière dont leur pratiques professionnelles en lien avec l’évaluation de la compréhension en lecture ont évolué suite à l’analyse des résultats et des outils d’évaluation relatifs à l’étude PIRLS 2011.
Regarder la capsule 1 en ligne
2. Des changements de pratiques professionnelles : des étudiantes du Master en Sciences de l’éducation décrivent la manière ils ont mis en place des cercles de lecture dans leur classe afin notamment de développer des stratégies de compréhension en lecture chez leurs élèves.
Regarder la capsule 2 en ligne
3. Les processus de compréhension qui ont constitué des obstacles pour les élèves de quatrième année primaire lors de l’enquête internationale PIRLS 2011 : ces processus sont explicités au départ du texte à visée narrative intitulé « La Tarte Anti-Ennemi » de Derek Munson. Certaines questions relatives au texte sont analysées en termes de processus de compréhension à mettre en œuvre, ceux-ci étant mis en lien avec les compétences telles que formulées dans le référentiel officiel des Socles.
Regarder la capsule 3 en ligne
4. Les résultats d’une analyse des réponses produites par des élèves de 4e année primaire aux questions relatives au texte intitulé La Tarte Anti-Ennemi. Cette analyse d’erreur éclaire la nature des difficultés éprouvées par les élèves pour aller au-delà d’une compréhension littérale, interpréter ou encore apprécier ces textes.
Regarder la capsule 4 partie 1 en ligne
Assessing the language competences of secondary education students
In 2005, the European Commission decided to organize a language competence assessment. This process is monitored by a Consultative Board made up of country representatives; a consortium, named “SurveyLang” and created for that purpose, has been put in charge of the management and coordination of the survey by the European Commission. The French Community decided, like 16 other countries or education systems, to take part in this study and put the aSPe in charge of its implementation, in collaboration with the Centre Approches quantitative des faits éducatifs (Quantitative aspects of educational facts) of the University.
This survey is largely inspired by the Pisa assets and sets up high quality standards in all its aspects, and stringent quality assurance mechanisms are applied from translation to data collection.
The assessment concerns reading, listening and writing competences in two (non mother) languages per country. The tests were designed according to the competence level grid supplied in the Common European Framework of Reference (Council of Europe, 2005). The competences defined on this basis are associated with specific tasks to enable comparability of results between the different languages. First, a routing test was administered and marked, to take into account the differences between individual competences and confront the students with an appropriate difficulty level test. Principals, language teachers, sampled students and the national project manager had to complete contextual questionnaires to supply data on the context where students develop their competences. These questionnaires also help to adjust the interpretation of the results according to significant variables such as the year number of the language learning or the magnitude of the language exposure. There were two versions of the tests: paper-based or computer-based tests.
In the French Community, the assessment was taken by students in 4th year of secondary education and concerned English and German. Because of the organization issues generated by the paper-based version of the tests (various levels of difficulty applied in most schools), the tests of the main campaign were administered via the computer mode. After a preparation phase and a field trial, the main campaign was carried out in early 2011 and the results were released at the end of 2012.
Après une phase préparatoire et un essai de terrain, l’enquête définitive a été réalisée début 2011 et les rapports sont attendus à partir de 2012.
More info
In favour of exploiting at school the diversity of European contexts
This study mainly aimed at a better knowledge of the attitudes and resources of young people at the age of compulsory schooling as far as learning languages is concerned. Thanks to the European Commission support, it was conducted in 5 European countries: Germany, Belgium (also in charge of the project coordination), Greece, Luxembourg and Poland. More than 6 000 young people completed a paper questionnaire, with about 300 students in grade 5 (5th year of primary school in Belgium) and 300 in grade 9 (3d year of secondary school in Belgium) in each tested region. The project presented effectively a methodological originality consisting in defining different areas, each one with a relative homogeneity of its linguistic environment, so that a quantitative study could be implemented, the results of which being also qualitatively interpreted in reference to their context.
Furthermore, in each area, policy-makers and resource people were questioned and supplied with information about the characteristics, resources and issues of each context. Three main aspects were brought out from the survey results: very often, a multilingual environment but with few languages taught, positive attitudes towards language learning, but differences among students (in favor of girls, students in academic tracks, students from privileged socio-cultural background, etc.), a true interest for languages spoken throughout the world but some hierarchy (beyond the very widespread languages – English, German, French -, the Romanic languages – mainly Spanish and Italian – are the most valued).
More info
References
Androulakis, G., Beckmann , C., Blondin, C. et al. (2007). Pour le multilinguisme : Exploiter à l'école la diversité des contextes européens : Résultats d'une étude internationale. Liège : Editions de l'Université de Liège.
http://orbi.ulg.ac.be/handle/2268/7582
Blondin, C., Fagnant, A. et al. (2007). Pour le multilinguisme : Exploiter à l'école la diversité des contextes européens : Analyse de trois zones belges dans le cadre d'une étude internationale. Liège : Editions de l'Université de Liège.
http://orbi.ulg.ac.be/handle/2268/11060
Goffin, C., Fagnant, A. et Blondin, C. (2009). Les langues des voisins : des langues toujours appréciées (2009). Revue de Linguistique et de Didactique des Langues, Vol. 40, 17-30.
http://orbi.ulg.ac.be/handle/2268/7566
Blondin, C., Fagnant, A. et Goffin, C. (2008). L'apprentissage des langues en Communauté française : curriculum, attitudes des élèves et auto-évaluation (pp. 73-92). Education-Formation.
http://orbi.ulg.ac.be/handle/2268/2996
Fagnant, A. et Blondin, C. et al. (2008). Image des langues et motivation. Langues Modernes Vol./Tome 3.
http://orbi.ulg.ac.be/handle/2268/3040
Share this page
