Evaluation

The ICC implements a series of procedures and tools to measure IFADEM’s state of advancement and its results; and to inform francophone institutions and partners of the initiative.

 

Évaluations externes (2008-2010)

Une double évaluation externe de la phase d’expérimentation menée au Bénin et au Burundi a eu lieu en 2010. Le cabinet ATEMA pour l’OIF et des experts désignés par le Conseil scientifique de l’AUF en ont été chargés.

 

 

Évaluations par les enseignants et les tuteurs

À l'issue du parcours de formation, les enseignants et les tuteurs sont été sollicités pour répondre à une enquête de satisfaction, anonyme, qui porte sur les différents aspects de l'Initiative : dispositif, tutorat, contenus, pratiques de classe, informatique, etc.

 

 

As a multi-actor organisation, IFADEM is subject to differing monitoring-evaluation modalities since every actor applies individual rules. Modalities and measurements for the evaluation of IFADEM may be divided into three groups:

  • Monitoring of the state of advancement and internal assessment;
  • Assessment of teachers’ needs and experience;
  • External assessments. 

Monitoring the state of advancement and internal evaluation

Archiving of Documents  

Agreements, reports

Production of activity reports by the ICC 

  • Monthly activity reports submitted by local teams; bi-monthly electronic newsletter published by the ICC;
  • Contractual reports stipulated by agreement : e.g. semester activity reports for the multi-country aid plan financed by AFD [French Development Agency—Agence française de développement], quarterly reports for Benin’s ministry of early childhood and primary education or for APEFE [association for the promotion of education and training abroad] in DRC, etc;
  • Reports of the coordination committee to francophone institutions, to the initiative’s coordination bodies (steering committee, partners group, advisory group), to francophone summits.

Monitoring and internal evaluation by the ICC and the advisory group

  • Monitoring by the Coordination committee of a country calendar, listing the steps of a phase-1 experiment and observer missions in the field;
  • Monitoring by the members of the advisory group, entrusted with scientific monitoring and monitoring assessment for IFADEM’s governing body, and the application of an experiment in a country; particularly in its early stages (participation in a co-construction seminar with the ministry at the start of the process), halfway through training, at the information seminar at the end of the phase-1 experiment. Reports submitted to the ICC;
  • Participation by members of the advisory group in the evaluation of a phase-1 experiment, when this evaluation is not stipulated by an agreement;
  • Survey questionnaires in a country at the end of a training programme for teachers and instructors. The questionnaire covers all teachers of a phase-1 experiment, and a representative selection for a phase-2 deployment. Quantity and quality analysis by a university laboratory. Publication on the website. Possibility of extending the process to other actors: headmasters, school inspectors. Information seminars;
  • At the end of a phase-1 experiment or a phase-2 deployment, IFADEM organizes a final seminar to allow a country’s actors to give their views (content developers, intermediary educational bodies, ministry) and beneficiaries (teachers).

Production of quantitative indicators

  • Monitoring of activities related to the implementation of a training programme: number of instructors trained, number of digital spaces, number of computers installed, number of handbooks produced and printed, number of teachers in training, number of certified teachers, etc.
  • Monitoring of more transversal activities: number of connections to the website, participation in social networks, number of published articles about the initiative, partnerships formed, etc.

Evaluation of teachers’ needs and knowledge

The ICC steers the conception and implementation of original measurement tools designed to evaluate the needs of teachers concerned by the initiative, and the impact that training has on them.

French-language placement tests 

Tests are employed before beginning a training course on a representative sampling of teachers. A contextualized placement test, in partnership with the International Centre for Educational Studies [Centre international d’études pédagogiques - CIEP], designed to evaluate the continuous training needs of teachers in French, was designed for Burundi and must be adapted to other countries. It may be used at the beginning and end of a training programme.

The TCF (knowledge of French test) is also used (Benin, Burundi, Haiti) to help evaluate the level of teachers in French, and adapt IFADEM training handbooks to their needs. IFADEM may also use tests results as a reference.

Formative and summative evaluations

  • Training handbooks: especially designed to make a diagnostic evaluation of a teacher’s level and knowledge of subject matter at the beginning of a training course (playing the same role as placement tests);
  • Handbooks and tutorials: formative evaluation is used to help the teacher in the training course. Handbooks are designed to measure progress. Instructors make regular verifications in face-to-face encounters;
  • Final tests: the partner ministry tests a teacher’s command of subject matter and competence with a certification test. The exam is theoretical (based on MCQs) and practical (class inspections), and must be passed by all teachers participating in the training course.

Measuring the professional competence of teachers

The ongoing design of an original tool to measure the professionalisation of teachers is composed of two parts :

  • Part 1: a questionnaire at the beginning and end of the training course, submitted to all teachers enrolled in a phase-1 experiment and a representative sampling in a phase-2 deployment. This questionnaire is designed to measure the evolution in teachers’ representations of their profession and practices in class.
  • Part 2: observation of professional practices in class at the beginning of the course and toward the end of the training period, to gauge possible changes in teaching practices.

Inspectors use an observation chart on a representative sample, in phase-1 experiments and phase-2 deployment.

The tool is not designed to evaluate educational systems, but is used by the ICC to verify the impact of the training course. The tool contributes to gauging IFADEM’s efficiency (if a teacher’s acquisition of knowledge is seen to progress) and equity (to try to limit and reduce the differences in level seen among teachers at the start of the programme).
 

External Evaluations

IFADEM is subject to a series of external assessments of its activities.

External evaluations of phase-2 deployment

Agreements provide for evaluations at the end of phase-2 deployments. An international tender is launched for the selection of experts, on the basis of terms of reference elaborated by the ICC, approved by the ministry and the technical and financial partner concerned. External evaluation operates on two levels: one, retrospective, measuring the results of the initiative and the other, prospective, helping to integrate IFADEM into a country’s system of continuous training.

Financial Audits in phase-2 deployment

An external auditor makes financial audits. They are selected from tenders and on the basis of terms of reference approved by the ministry and the technical and financial partner concerned. They are provided for in agreements on phase-2 deployment. The audit checks terms of commitment of funds provided by a technical and financial partner and gives advice about the implementation of budget management.

Additional external evaluations 

An external assessment of the AFD support plan (“CZZ 1588”) is projected for early 2016 and an assessment by the AUF scientific council still to be determined.

In the context of an evaluation of the AFD multi-country aid plan, an international tender will be launched in 2013 to determine the terms of measuring the impact of the initiative upon teachers, and potentially students. A few countries will be targeted. 
 

 

Submitted on Last update on