M & E Indicators

Monitoring is the systematic supervision of activities in progress to ensure that they are on curse and schedule in meeting objectives and performance targets. It is a test or sampling on a regular or ongoing basis of a system, program, project or activity for any changes in application performance, transaction, problems and potentials for changes as well as the effectiveness and efficiency. This involves a systematic collection and analysis of data as a project progresses. Generally, monitoring involves: • Establishing indicators of efficiency, effectiveness and impact • Setting up systems to collect information related to these indicators • Collecting and recording the information • Analyzing the information • Using the information to inform day-to-day management Evaluation is the systematic determination of the merits, worth and significance of something or following some certain set standards, an assessment of the degree to which a program, project, system or activity fulfills stated objectives and goals. This requires a comparison of actual project impact against the agreed strategic plans. Generally, evaluation involves: • Looking at what the project or organization intended to achieve • Assessing its progress towards what it was made to achieve • Looking at the strategy of the project or organization • Looking at how it worked The bottom-line is that both monitoring and evaluation all focus on efficiency, effectiveness and impact of projects, programs or activities and through the two processes, we review progress, identify problems and potentials in planning and implementation, suggest possible solutions, raise questions, stimulate some reflection on the goals and objectives, provide information on insight and make necessary adjustments for improvement. Objectives of Program Evaluation • To inform decision on operations, policy, or strategy related to ongoing or future program interventions • To promote accountability and transparency to stakeholders, decision-makers and donors • To enable corporate learning and contribute to the body of knowledge on what works and what does not work and why • To verify and improve program quality and management • To identify successful strategies for extension, expansion and replication • To modify unsuccessful strategies • To measure the effects and benefits of programs and project intervention • To give stakeholders the opportunity to have a say in program output and quality • To justify and validate programs to donors, partners and constituencies • To enhance the generation and use of value-added evaluative information The American Evaluation Association (AEA) has some specific Guiding Principles established for various evaluation exercises such as Systematic Inquiry, Competence, Integrity and Honesty, Respect for People and Responsibilities for the General and Public Welfare. 1- Systematic Inquiry- Evaluators conduct systematic data-based inquiries 2- Competence- Evaluators provide competent performance to stakeholders 3- Integrity and Honesty- Evaluators display honesty and integrity in their own behavior and attempt to ensure the honesty and integrity of the entire evaluation process 4- Respect for People- Evaluators respect the security, dignity and self-worth of respondents, program participants, clients and other evaluation stakeholders 5- Responsibilities for the General and Public Welfare-Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation. Characteristics of Monitoring and Evaluation Monitoring Evaluation Continuous Periodic: at important milestones such as mid-term program implementation; at the end of program such as final report Keeps track: oversight, analyses and document progress In-depth Analysis; Compares planned with actual achievements Focuses on inputs, activities, outputs, implementation processes, continued relevance, likely results at outcome level Focuses on outputs in relation to inputs; results in relation to costs; processes used to achieve results; overall relevance; impact and sustainability Answers what activities were implemented and results achieved Answers why and how results were achieved. Contributes to building theories and models for change Alerts managers to problems and provides options for corrective action Provides managers with strategy and policy options Types of Evaluations 1- Formative Evaluation Formative Evaluation is exercised when new programs, new interventions, new procedures or new elements of existing programs are proposed. It is often during the pre-implementation or design phase of the project and emphasize needs assessment 2- Process Evaluation Process Evaluation involves documenting actual program functioning and identifying barriers to implementation. This has to do more with program monitoring. This includes the identification of target population, description of services delivered, the use of resources and the qualifications and experiences of personnel working in them. 3- Outcome Evaluation Outcome Evaluations are carried out to measure the effectiveness of a program and as such deal with program output or results. Criteria used are objectivity, measurability, attributable and sensitivity. 4- Cost-Effectiveness Analysis This evaluates the cost of the program against its effectiveness and efficiency in terms of targeted population or client served, services rendered and cost etc. 5- Cost-Benefit Analysis This evaluation examines the financial, economic, social, moral, psychological and other costs of the program against its economic, financial, social, moral, psychological and other benefits. Indicators of Monitoring and Evaluation A-Economic Development Indicators  Average annual household income  Average weekly/monthly wages  Employment, by age group  Unemployment, by age group, by gender  Employment, by occupation, by gender  Government employment  Earned income levels  Average length of unemployment period  Default rates on loans  Ratio of home owners to renters  Per capita income  Average annual family income  % people below the poverty line  Ratio of seasonal to permanent employment  Growth rate of small businesses Social Development Indicators  Death rate  Life expectancy at birth  Infant mortality rates  Causes of death  Number of doctors per capita  Number of hospital beds per capita  Number of nurses per capita  Literacy rates, by age and gender  Student: teacher ratios  Retention rate by school level  School completion rates by exit points  Public spending per studnet  Number of suicides  Causes of accidents  Dwellings with running water  Dwellings with electricity  Number of homeless  Number of violent crimes  Birth rate  Fertility rate  Gini distribution of income (see Glossary of Terms)  Infant mortality rate  Rates of hospitalisation  Rates of HIV infection  Rates of AIDS deaths  Number of movie theatres/swimming pools per 1000 residents  Number of radios/televisions per capita  Availability of books in traditional languages  Traditional languages taught in schools  Time spent on listening to radio/watching television by gender  Number of programmes on television and radio in traditional languages and/or dealing with traditional customs  Church participation, by age and gender C- Political/Organisational Development Indicators  Number of community organisations  Types of organised sport  Number of tournaments and games  Participation levels in organised sport  Number of youth groups  Participation in youth groups  Participation in women’s groups  Participation in groups for the elderly  Number of groups for the elderly  Structure of political leadership, by age and gender  Participation rate in elections, by age and gender  Number of public meetings held  Particiaption in public meetings, by age and gender Difference Between Monitoring and Evaluation CRITERIA MONITORING EVALUATION Objective To track changes from baseline conditions to desired outcomes To validate what results were achieved and how and why they were or not achieved Focus Focuses on output of projects and programs Compares planned with intended outcomes achieved. Focuses on how and why outputs and strategies contributed to achievement of outcomes. Focuses on question of relevance, effecitvieness, sustainability and chamge Methodology Tracks and assesses performsnce through analysis and comparison of indicators over time Evaluates achievement of outcomes by comparing before and after the interventions. Relies on monitoring data to information from external sources. Conduct Continuous and systematic by Task Managers, Project Managers, Program Managers and key partners. Time-bound, periodic, in-depth. External evaluators and partners Use Alerts manager to problems in performance, provide options for corrective actions and helps demonstrate accountability Provides managers with strategy and policy options, provides basis for learning and demonstrates accountability. Designing the Monitoring and Evaluatiojn Process Monitoring and Evaluation designs may vary depending on the type of information to be collected, the purpose of the evaluation, the nature of the project and the budget available. They can be Quantitative, Qualitative, Experimantal, Quasi-Experimental and Descriptive designs. Quantitative designs methods explain ‘what’, ‘how much’, or ‘how many’. These deal often with numbers. Qualitative designs answer ‘why’ and ‘how’ questions. When evaluating innovative projects and programs such as technology applications, qualitative designs are more appropriate Experimental and Quasi-Experimental designs try to estimate the effects and impact of projects and programs Descriptive designs try to describe what happened in terms of activities, outputs and performance.