Standardized Monitoring and Assessment of Relief and Transitions (SMART)

An Inter-Agency Initiative



Expert Panel Meeting on the SMART Methodology

July 13 – 15, 2004:  Brussels, Belgium

Centre for Research on the Epidemiology of Disasters (CRED)



The Centre for Research on the Epidemiology of Disasters (CRED) organized the Expert Panel meeting to discuss the development of the SMART Methodology, July 13-15, 2004, Brussels, Belgium. This was part of the consultative process to review the components and options for triangulating nutritional status of children under five, mortality rate, and food security in an integrated survey protocol. The Expert Panel agreed on the technical parameters for combining the assessment of Nutritional Status of Children Under-Five, Crude Mortality Rate (CMR) and food security in an integrated survey protocol. The critical set of core information for each component was outlined which will be used as the base for developing the methodology. The Expert Panel reviewed and endorsed the windows-based analytical software which simplifies data entry and analysis, with quality-assurance checks built into the software program. It reviewed several pilot countries for testing the methodology under various field conditions. It established an inter-agency work-plan which is to have a draft manual ready by end August to be reviewed by the Expert Panel and others, finalized by end September, and then pilot-tested. A meeting of all G-8 donors and other stakeholders will be convened before end 2004 to roll-out the SMART Methodology.


The development of the SMART Methodology is a follow-up to the July 2002 SMART workshop that established a broad-base consensus that Crude Mortality Rate (CMR) and Nutritional Status of Children Under-Five are the most vital, basic public health indicators of the severity of a humanitarian crisis. They are useful for assessing need and prioritizing resources. They monitor the extent to which the relief system is meeting the needs of the population and thus the overall impact of the relief response. The 2002 workshop recommended the development of a generic, standardized methodology and the “SMART Manual for Dummies” and other tools for all humanitarian organizations.[1] The Canadian International Development Agency (CIDA) provided funds for developing the SMART Methodology and pilot-testing in one country. 

With the adoption of the SMART Initiative by the G-8 Action Plan on Ending Famine,[2] the development of the SMART Methodology and related activities is a priority of donors and participating organizations.  With multi-partner buy-in, SMART is an important focus of G-8 member support for improved worldwide emergency assessment and response systems. G-8 donors are advancing SMART by providing funds for priority actions, participating in technical reviews and meetings, advocating the use of the SMART Methodology in other fora, including the Good Humanitarian Donorship (GHD) Initiative and working with international, non-governmental organizations and humanitarian coordination groups to encourage the adoption of the SMART Methodology.

The following is a summary of the Expert Panel meeting presentations.  PowerPoint presentations and discussion notes are posted at


SMART Methodology (A. Ralte)

The SMART Methodology will help to determine needs, monitor and report on trends and the impact of humanitarian aid. This will be undertaken using (for the first time) standardized data collection, analysis, interpretation and reporting. It is envisioned as a complete package of a survey protocol, guidelines, and windows-based analytical software.

The audience for the SMART Methodology is all humanitarian organizations, and SMART partners, including host governments. The level of technical difficulty for the methodology is targeted to PVOs/NGOs with technical background and/or training.  As noted at the SMART workshop, July 2002: “The methodology should be technically sound and simple, that is do-able by our partners, in particular PVOs/NGOs. This should be the balance.”   

The SMART Methodology will have the following elements: 

SMART Version 1 will be based on anthropometry (nutritional status assessment), with the most critical components of mortality (CMR) and food security (HEA “lite” version) integrated.  As recommended at the July 2002 workshop, it will be based on viable, current best-practice methodologies until further research determines change.


Integrated Concept (J. Seaman)

The July 2002 workshop indicated that nutritional survey data cannot be interpreted in isolation, and that the food security context needs to be understood. The interpretation of nutrition data is difficult and food security helps put this in context.

The Household Economy Approach (HEA) (Livelihood method) is the only approach that can meet SMART Version 1 criteria. Although HEA is data-heavy and needs intensive training and know-how, HEA “lite” version that uses information from key informants (farmer, tax official, etc.) is an option for SMART Version 1.  The critical set of information need is the main features of the economy (usually limited to two to three activities)with assumptions or uncertainties clearly presented in the methodology. HEA started with the “lite” version and then got “heavy” – the essence is in the data structure, and what questions to ask – and, therefore, key informants can be used to construct the basic data set. 

The aim is a “joined-up story” that triangulates nutritional status and food security, and with mortality rate. This data will help to tell the “story” as the SMART indicators are considered basic well-being indicators. Food quantity and quality are the basic drivers of mortality and triangulated data should correspond -- if they do not, this should be an alert for the community to investigate what is happening to the population. 


Nutrition Assessment (M. Golden)

Mortality rate is the most basic public health vital statistic. It gives an overall measure of the severity of a situation. Wasting and edema rates are the basic measures of nutritional health.

There is a need for both quantitative and qualitative assessments in emergencies, and a need to triangulate data. Surveys must be as simple as possible while providing reliable information. The survey instrument should collect essential but not excess data, and it must not have a long questionnaire - every question added means a compromise on data quality.

We need to systematize how to resolve practical problems and constraints, such as security and arrivals from inaccessible areas, population movement, scattered population, and topography. There is also difficulty in determining age – should we use 110 cm cutoff instead? Measurement errors are rampant due to inadequately trained staff and inaccurate population figures – and most survey data are likely to be overestimates. This is why transparency is needed, including a standard reporting format for all humanitarian organizations.

The critical set of quantitative information needed includes: wasting rates, edema rates, Crude Mortality Rate (CMR), population size, and demography. It also needs context and food security data for interpretation of the quantitative data.  A survey can tell us the current level of severity and help to calibrate surveillance data but contextual data is needed to interpret trends or changes. It is also important to note that the diet can be poor and there are many other nutritional deficiencies (vitamin A, iron, iodine, vitamin C, riboflavine, B12, folate, niacin, etc.) that do not cause wasting or edema but can lead to immuno-incompetence and death. See the illustration of the pellagra problem in Angola (PowerPoint presentation).

A review of 400 surveys (increased from 223 surveys reviewed at the time of July 2002 workshop) show that the standard deviation does not change as the population becomes more malnourished. As populations become malnourished, all individuals are affected to a similar extent and the whole distribution moves downwards – and weight loss is the same among malnourished and well-nourished children. Movement in weight-for-height is not necessarily a “trailing indicator” in population terms, and it is a sensitive indicator of stress within the population. Who becomes malnourished may relate more closely to location than other vulnerability indicators. Since vulnerability seems to be geographically determined, geographical targeting is recommended as it is cheaper and more efficient than identifying and targeting individual households.

Basic elements to be considered in anthropometry:

Issues for further discussion and/or research:


Food Security (J. Seaman)

A livelihood approach has been proposed as this meets the SMART conceptual framework and practical criteria – this will allow a PVO/NGO to explain anthropometric survey and mortality findings, where the PVO is operating at a reasonably local scale. However, existing livelihood approaches, such as the SCF/UK Household Economy Approach (HEA) have potentially large data and skill requirements which are impractical for most PVOs. Therefore, some simplification is required for the SMART Methodology.

The basic logic of livelihood methods (based on A. Sen’s entitlement concept) is to understand the non-crisis economy of a defined population in sufficient quantitative and qualitative detail to be able to estimate the impact of a defined shock on that economy.  Specifically, it is to estimate the ability of people in different wealth groups (poor, better-off) to acquire sufficient food to meet their needs, where non-food expenditure (e.g., fuel, health costs) is specified.

The Household Economy Approach (HEA) was originally developed, and has chiefly been used for predicting the impact of a shock, e.g., crop failure from drought, a price rise, on food security – at middle to large scale (e.g., district – national).  This requires substantial inputs for data collection and a fully quantified analysis.  Household data (a household budget and assets) is obtained for a “typical” household for each (or at least three) wealth groups plus contextual information (such as availability of wild foods, etc.).  Data collection is standardized.

Analysis is undertaken by a simple arithmetic simulation, which allows the user to work systematically through a case and develop a hypothesis or hypotheses about the likely impact of a “shock” (to production, prices, etc.) on the households concerned.  This works in two steps: 1) an estimate of the direct impact of a shock on food access by household type, and 2) an estimate of the ability of the household to compensate for any deficit in step 1. The model can be expressed on a spreadsheet, or using software (RiskMap).  HEA is widely used and has proved to be technically effective. (For illustration, see PowerPoint presentation).  The output of an HEA analysis is a logical argument about the most likely connection between a shock and people’s future ability to get sufficient food, where the methods, assumptions and uncertainties are clearly stated.

HEA “lite” (proposed for SMART) uses the basic structure and logic of HEA, i.e., to understand the impact of a shock on an economy.  However, information is obtained from key informants rather than by primary data collection, and there is less emphasis on detailed quantification.  The aim is to have sufficient information to set nutrition findings in a context, to allow a prediction about likely future changes in nutrition, food needs and relief action.  These data techniques have been used by SCF/UK for rapid updating of existing data, e.g., Malawi 2002. (See PowerPoint presentation).

The aim is to put together a “joined up story” about:

This should be consistent with and help “explain” nutrition survey findings and mortality. All assumptions and uncertainties should be clear to the end-user.

Issues to be resolved for SMART Methodology Version 1:


Mortality (C. Robinson, D. Guha-Sapir)

Many different methods are available to measure mortality such as observations (body counts, burial site activity), participatory appraisal (use of key informants, community mapping or locating health clinics and places of workshop), surveillance (passive – clinic data; active – burial sites, body counts, neighborhood interviews), and surveys. There are three approaches currently used in surveys. These are:

Issues to consider when designing surveys:

What is the context/setting in which we are gathering information? Is this for response or for the record?

What level of precision do we need?

What period of reporting or recall is used? This should be stable and consistent from one survey to the next.

What events contribute to household change, exposure time? How to define household, family?

What kinds of death may be more critical to the life of the population?

How frequently should the surveys be carried out?


Issues to be resolved for SMART Methodology Version 1:



Integrated Software Package (J. Erhardt)

A prototype windows-based software has been developed that is simple and flexible for data entry and analysis, with quality assurance checks built into the program. This can be easily translated into other languages. The prototype software is available at   

Technical support will be built in as part of the SMART training and capacity-building. While the software should be free to all organizations, some participants suggested incorporating a maintenance program and user’s fee for de-bugging and upgrading with contributions from users.

Issues to be resolved for SMART Methodology Version 1:


Complex Emergencies Database (CE-DAT) (D. Hargitt)

The objective of CE-DAT is to support decision making and promote effectiveness of prevention and response. There are 8 pilot countries (Afghanistan, Angola, Democratic Republic of Congo, Iraq, Ivory Coast, Rwanda, Sierra Leone, Sudan), and 3 main categories of data (mortality, malnutrition, morbidity). The objective for the next few months is to integrate CE-DAT format with the SMART software, mapping the information, increasing the number of countries and categories, obtain data directly from data source, and use of optical readers for data entry. A reliability scale should be set up for all data on CE-DAT.


Vulnerability: Update on G-8 Discussion (J. Becker) 

SMART contributes to larger undertakings that include the G-8 process and initiatives, USAID data policy issues, and other assessment and response initiatives.

A.         G-8 Process and Initiatives:

The three U.S. Initiatives under the G-8 Africa and Famine Action Plans are:

The G-8 Statement on SMART is reflected under Worldwide Emergency Assessments and Response Systems.

 B.            Assessment Initiatives (outside the G-8 Process) Relevant to SMART:

(See PowerPoint presentation for details)

Follow-up: A draft scope of work for the Expert Panel on Vulnerability Assessment will be developed and shared with participants and others.


Operational Research (N. Mock)

 The session focused on two objectives with emphasis on the first:

Related to the first, the group agreed that there are a number of knowledge gaps that constrain our ability to identify and standardized methods of assessment in complex emergency and post conflict settings. While these gaps do not prevent the launching of SMART Version 1, they are important for further refinement of methods and information use. Priority research themes were reviewed – see PowerPoint presentation for details.

The Expert Panel also concluded that a Task Force for Research (to be established) will convene, probably, virtually to further elaborate, prioritize research and hopefully solicit funding for specific proposals. Nancy Mock agreed to initially coordinate the Task Force. She will solicit membership by email as a follow-up to this meeting.


Pilot-Test of SMART Methodology (A. Ralte)


SMART will be applied in a pilot-country to see if the methodology and the guidelines work under field conditions. The pilot-test will determine ease of use of the methodology and guidelines, and whether any further modifications or fine-tuning will be needed before roll-out of SMART Methodology Version 1.

Participants at the July 2002 workshop had discussed Afghanistan and DRC, and selected Angola as the initial pilot country. Since then because of the G-8 focus, Ethiopia is being considered. Countries such as Ethiopia and Angola where national guidelines are in place make for good candidates. Burundi was recommended due to host government commitment and being a small country might facilitate coordination.  Other suggested countries include:

Rwanda – represents a government in transition

Chad - country with high mortality rates

Liberia - where CDC is undertaking population-based maternal mortality study

Malawi - where large-scale household food security assessment has just been completed which might make it useful for comparing the HEA “lite” methodology

Ivory Coast - country currently in emergency phase which contradicts another suggestion to start in a country that is not going through an acute emergency

Follow-up: Recommendations will be reviewed and final selection will be based on further consultations with organizations/individuals unable to participate at the meeting.


Next Steps

Donors (and others) should continue to advocate that funding priorities should be evidence-based and encourage the use of good quality data for making decisions. This will also help tell the story of how humanitarian assistance is impacting lives. Donors should harmonize on assessments, survey and reporting, and develop joint policy/operational guidelines. The SMART Methodology should be a useful tool for practitioners so that all organizations will want to use it voluntarily. Outreach to other G-8 donors (not yet fully on board) and other groups is also an important follow-up.

A.         Issues for Further Deliberations:

B.         SMART Methodology Work Plan:

[1] See

[2] “Building on the consensus reached in the Standardized Monitoring and Assessment of Relief and Transitions (SMART) Initiative, the G-8 will support further activities to improve needs assessment and monitoring of famine and food security. This will include the establishment of a multi-partner experts’ panel to review standards of practice for vulnerability assessments and food security and the development of on-line information systems to disseminate information on vulnerable areas, needs assessments, and the impact of assistance operations.”