Fortify Health enables access to micronutrient-rich wheat flour in order to reduce and prevent iron-deficiency anaemia. A key component of our work is providing support to millers in India to fortify the wheat flour that they produce for the open market.
Understanding the impact of our open market work is critical to Fortify Health’s mission. We claim to be a data-driven organisation that values impact, but this would be meaningless if we did not invest in understanding and evaluating our own impact. This post seeks to explain how we view our own impact at Fortify Health and provide information to our followers about the data we rely on and calculations we make in that process.
We cover the following topics:
We are indebted to a huge number of different advisors and followers for advancing our understanding of our impact in recent years. In particular, GiveWell has been a crucial partner for Fortify Health in advancing our understanding of our own intervention.
Section 1: What is the problem that we are addressing?
We assess the prevalence of anaemia in the areas where we operate by using widespread survey data and applying adjustments to account for state-based and socioeconomic variations.
The first element of analysing the impact of Fortify Health’s open market work is identifying and sizing the problem that we aim to solve. Fortify Health’s programme targets anaemia caused by iron deficiency, a widespread issue in India with significant health and economic repercussions. We believe that there are three major effects of anaemia that are relevant when analysing the impact of chakki atta fortification:
Establishing the Prevalence of Anaemia
The starting point when sizing all three of these effects is to establish the prevalence of anaemia in the regions of India where we work. There are a range of different potential datasets and surveys that could be used to do this (and there are significant debates about which survey is the best representation of anaemia prevalence). We tend to rely on one of the more conservative sources, the Comprehensive National Nutrition Survey from 2019 (the CNNS), to estimate prevalence when considering our impact. The CNNS indicates that:
CNNS is only from 2019, however, and the figures given above are nationwide (not for the specific states that Fortify Health works in). The following factors are therefore relevant when determining how to apply the CNNS to Fortify Health’s open market work:
Morbidity Impacts of Anaemia
We use this method across all three primary effects of anaemia. However, it is necessary to make some further adjustments for morbidity, in particular. We use a concept called Years Lived with Disability (YLD) to quantify morbidity impacts. A YLD is a concept developed by health economists to quantify how much a disease like anaemia affects a person's life. Think of each year lived with anaemia as a year where your quality of life is a bit lower because you're not as healthy.
Section 2: How effective is chakki atta fortification?
We rely on several relevant studies to assess the effectiveness of chakki atta fortification in reducing iron-deficiency anaemia and apply adjustments to reflect Fortify Health’s specific context.
The second element of analysing the impact of Fortify Health’s open market programme is to consider the effectiveness of what we do. Fortify Health’s open market programme seeks to combat the high prevalence of iron-deficiency of anaemia by supporting mills in India to add iron, vitamin B9 and vitamin B12 to wheat flour, which is commonly eaten. Our goal is to reduce the number of people suffering from anaemia and, in turn, to reduce the three primary negative effects of iron-deficiency anaemia (anaemia morbidity, cognitive impairment in children and cognitive impairment in adults).
This section explains how we evaluate the effectiveness of our open market programme in achieving these goals.
Impact of Fortification on Anaemia Prevalence
The causal relationship between chakki atta fortification and the reduction of anaemia is absolutely critical to understanding the impact of our open market intervention.
We tend to rely on an authoritative meta-analysis conducted in 2021 (Field et al.) to assess the effectiveness of wheat flour fortification in decreasing the prevalence and severity of anaemia in the population. Meta-analyses are particularly valuable for assessing the effectiveness of interventions because they combine and analyse multiple studies, providing a more comprehensive and statistically robust view. Field et al. (2021) provides a high-level summary of six similar studies and concluded that the risk ratio for anaemia with wheat flour fortification is 0.73, suggesting that the intervention results in a 27% reduction in anaemia prevalence.
We use this risk ratio from Field et al. as a base when considering our own impact, but we then make further adjustments to reflect uncertainties around how the conclusions from Field et al. might apply to Fortify Health’s intervention. In particular, we think the following adjustments are relevant:
This leads us to conclude that Fortify Health’s open market programme could potentially result in a 28% reduction in anaemia prevalence in target populations (reflected as 0.28). We then apply this figure to the total annual YLDs caused by anaemia in India (0.010 YLDs, see Section 2) to conclude that 0.0029 YLDs could be averted by iron fortification per person per year by iron fortification.
Cognitive benefits for children and adults
Iron fortification not only reduces anaemia but also has the potential to boost cognitive function in children and adults. There is limited evidence about the exact effect of iron fortification in this regard, so we rely on studies related to iron supplementation to estimate that effect. We then apply appropriate adjustments to these estimates:
This analysis indicates that iron fortification could lead to an increase in IQ of around 1.3 points per child and 0.35 points per adult.
Section 3: Uncertainties
There is more work for Fortify Health to do in evaluating our own impact and we are continually working to address uncertainties.
Our evaluation of our impact does not take into account the effect of implementation difficulties, such as wastage or ineffective fortification. We monitor both of those things through our general quality and programme monitoring, seeking to minimise them as much as possible.
There are also several areas that we would like to explore in more depth in the coming years and months:
These uncertainties highlight the need for ongoing evaluation and consideration of diverse factors affecting the programme's impact. Understanding these limitations helps us refine our approach and ensure that our fortification programmes deliver maximum benefits to the target populations.
Did you enjoy reading this post? To learn more about our work, sign-up to our newsletter here.
Understanding the impact of our open market work is critical to Fortify Health’s mission. We claim to be a data-driven organisation that values impact, but this would be meaningless if we did not invest in understanding and evaluating our own impact. This post seeks to explain how we view our own impact at Fortify Health and provide information to our followers about the data we rely on and calculations we make in that process.
We cover the following topics:
- 1. the scale and effects of the problem we are seeking to address, iron-deficiency anaemia;
- 2. the effectiveness of our open market programme in addressing that problem; and
- 3. our remaining uncertainties or questions.
We are indebted to a huge number of different advisors and followers for advancing our understanding of our impact in recent years. In particular, GiveWell has been a crucial partner for Fortify Health in advancing our understanding of our own intervention.
Section 1: What is the problem that we are addressing?
We assess the prevalence of anaemia in the areas where we operate by using widespread survey data and applying adjustments to account for state-based and socioeconomic variations.
The first element of analysing the impact of Fortify Health’s open market work is identifying and sizing the problem that we aim to solve. Fortify Health’s programme targets anaemia caused by iron deficiency, a widespread issue in India with significant health and economic repercussions. We believe that there are three major effects of anaemia that are relevant when analysing the impact of chakki atta fortification:
- 1. Anaemia morbidity - iron-deficiency anaemia can have a significant negative impact on an individual’s quality of life, causing tiredness, weakness, dizziness or light-headedness, drowsiness, and shortness of breath. Morbidity refers to these ways that iron-deficiency anaemia can impact quality of life;
- 2. Cognitive development in children - iron-deficiency anaemia impairs cognitive development in children, leading to lower IQ and worse educational outcomes; and
- 3. Cognitive development in adults - iron-deficiency anaemia also impairs cognitive development in adults, with potential knock-on effects for productivity.
Establishing the Prevalence of Anaemia
The starting point when sizing all three of these effects is to establish the prevalence of anaemia in the regions of India where we work. There are a range of different potential datasets and surveys that could be used to do this (and there are significant debates about which survey is the best representation of anaemia prevalence). We tend to rely on one of the more conservative sources, the Comprehensive National Nutrition Survey from 2019 (the CNNS), to estimate prevalence when considering our impact. The CNNS indicates that:
- 1. 35% of working-age adults aged 15-64 suffer from anaemia (and this age group makes up approximately 66% of the Indian population); and
- 2. 36% of children aged 5-14 suffer from anaemia (and this age group makes up 19% of the Indian population).
CNNS is only from 2019, however, and the figures given above are nationwide (not for the specific states that Fortify Health works in). The following factors are therefore relevant when determining how to apply the CNNS to Fortify Health’s open market work:
- 1. future trends - the CNNS suggests that anaemia in India is gradually getting better. It is therefore necessary to make some adjustment to the figures to account for future increases or decreases in prevalence;
- 2. socioeconomic factors - wealthier individuals who buy flour from the open market are slightly less likely to suffer from anaemia than the general population. We therefore apply a slight downward adjustment to the CNNS data to reflect this difference (based on data from the National Family Household Survey 4, from 2015-2016); and
- 3. state-level variation - government data indicates that anaemia is slightly more prevalent in the states that Fortify Health works in (Madhya Pradesh, Maharashtra) than it is across India. We therefore apply slight upward adjustments to the CNNS figures to account for these differences.
Morbidity Impacts of Anaemia
We use this method across all three primary effects of anaemia. However, it is necessary to make some further adjustments for morbidity, in particular. We use a concept called Years Lived with Disability (YLD) to quantify morbidity impacts. A YLD is a concept developed by health economists to quantify how much a disease like anaemia affects a person's life. Think of each year lived with anaemia as a year where your quality of life is a bit lower because you're not as healthy.
- 1. Total Anaemia Morbidity - In India, anaemia as a whole has a substantial impact and Global Burden of Diseases (GBD) study suggests that around 1,200 YLDs per 100,000 people in India are attributable to anaemia each year (equivalent to around 0.005 YLDs per person annually).
- 2. Iron-deficiency Anaemia Morbidity - The above figure takes into account all anaemias, however, not just iron-deficiency anaemia. A further adjustment is therefore needed to estimate what proportion of total anaemia cases are due to iron deficiency (and therefore addressable by food fortification). The GBD provided data in 2019 about the proportion of total anaemia cases attributable to dietary iron deficiency (around 60%).
Section 2: How effective is chakki atta fortification?
We rely on several relevant studies to assess the effectiveness of chakki atta fortification in reducing iron-deficiency anaemia and apply adjustments to reflect Fortify Health’s specific context.
The second element of analysing the impact of Fortify Health’s open market programme is to consider the effectiveness of what we do. Fortify Health’s open market programme seeks to combat the high prevalence of iron-deficiency of anaemia by supporting mills in India to add iron, vitamin B9 and vitamin B12 to wheat flour, which is commonly eaten. Our goal is to reduce the number of people suffering from anaemia and, in turn, to reduce the three primary negative effects of iron-deficiency anaemia (anaemia morbidity, cognitive impairment in children and cognitive impairment in adults).
This section explains how we evaluate the effectiveness of our open market programme in achieving these goals.
Impact of Fortification on Anaemia Prevalence
The causal relationship between chakki atta fortification and the reduction of anaemia is absolutely critical to understanding the impact of our open market intervention.
We tend to rely on an authoritative meta-analysis conducted in 2021 (Field et al.) to assess the effectiveness of wheat flour fortification in decreasing the prevalence and severity of anaemia in the population. Meta-analyses are particularly valuable for assessing the effectiveness of interventions because they combine and analyse multiple studies, providing a more comprehensive and statistically robust view. Field et al. (2021) provides a high-level summary of six similar studies and concluded that the risk ratio for anaemia with wheat flour fortification is 0.73, suggesting that the intervention results in a 27% reduction in anaemia prevalence.
We use this risk ratio from Field et al. as a base when considering our own impact, but we then make further adjustments to reflect uncertainties around how the conclusions from Field et al. might apply to Fortify Health’s intervention. In particular, we think the following adjustments are relevant:
- 1. Internal and external validity - the contexts covered by Field et al. have similar relevant characteristics to the context in which Fortify Health operates, so we do not think external validity is important. However, we believe that a small discount is necessary to cover the general risk that the study’s conclusions may not be reliable (also know as ‘internal validity’); and
- 2. NAFeEDTA - the iron compound used by Fortify Health’s partners, NAFeEDTA, is more bioavailable (i.e. more likely to be absorbed by the body) than the iron compounds used in some of the studies in Field et al.. We therefore apply a small upwards adjustment to the reduction identified in Field et al. to reflect the bioavailability of NaFeEDTA.
This leads us to conclude that Fortify Health’s open market programme could potentially result in a 28% reduction in anaemia prevalence in target populations (reflected as 0.28). We then apply this figure to the total annual YLDs caused by anaemia in India (0.010 YLDs, see Section 2) to conclude that 0.0029 YLDs could be averted by iron fortification per person per year by iron fortification.
Cognitive benefits for children and adults
Iron fortification not only reduces anaemia but also has the potential to boost cognitive function in children and adults. There is limited evidence about the exact effect of iron fortification in this regard, so we rely on studies related to iron supplementation to estimate that effect. We then apply appropriate adjustments to these estimates:
- 1. Internal and external validity - the contexts covered by supplementation-related studies differ substantially from Fortify Health’s and we see some risk of internal validity issues. We therefore apply substantial adjustments to account for these uncertainties;
- 2. Difference between fortification and supplementation - iron fortification is generally less effective than iron supplementation in reducing anaemia, because the doses delivered through fortification are smaller (see, for example, Low et al (2013)). We therefore apply an additional downward adjustment to these estimates to reflect the fact that Fortify Health’s programme is iron fortification, not supplementation. We apply a downward adjustment factor of 62.1% to the headline estimate to reflect the lower effectiveness of iron fortification compared to iron supplementation.
This analysis indicates that iron fortification could lead to an increase in IQ of around 1.3 points per child and 0.35 points per adult.
Section 3: Uncertainties
There is more work for Fortify Health to do in evaluating our own impact and we are continually working to address uncertainties.
Our evaluation of our impact does not take into account the effect of implementation difficulties, such as wastage or ineffective fortification. We monitor both of those things through our general quality and programme monitoring, seeking to minimise them as much as possible.
There are also several areas that we would like to explore in more depth in the coming years and months:
- 1. Other benefits of chakki atta fortification - what effect does Fortify Health’s support for B9 and B12 fortification have on our ultimate beneficiaries? Are there any other health benefits to wheat flour fortification that have not yet been identified?
- 2. Economic effects of anaemia - how does high prevalence of anaemia impact a country’s economic health? Are those impacts driven by cognitive impairment or other channels?
- 3. Consumption patterns - how much and how frequently do Fortify Health’s ultimate beneficiaries consume fortified chakki atta?
These uncertainties highlight the need for ongoing evaluation and consideration of diverse factors affecting the programme's impact. Understanding these limitations helps us refine our approach and ensure that our fortification programmes deliver maximum benefits to the target populations.
Did you enjoy reading this post? To learn more about our work, sign-up to our newsletter here.
