Data-free zones
The obvious question is why is it that we don’t have quality data?
The writer is a Howard Hughes Medical Institute professor of biomedical engineering, international health and medicine at Boston University. He tweets @mhzaman
When physician Amir Aman became the Minister of Health in Ethiopia, he noted something troubling. He knew how many deliveries happened in the country, and the total number of women giving birth. He also knew the maternal mortality rate. A simple calculation suggested that the total number of mothers who unfortunately die during childbirth annually should be around 4,000. But upon tallying the numbers from hospitals, clinics and other health outposts, the total being reported was about half that. He dug in, made calls, and visited health centres. Soon it became clear that the problem was in reporting and incentives for reporting. Health centres were underreporting (or in many cases not reporting at all) because there were incentives to show fewer deaths — even when that was simply not true. This was more than incentives. This meant that women and families were falling through the cracks, and the policies being created were never going to address the problem.
There was no incentive to be honest and truthful. Aman’s controversial policies incentivised accurate reporting, and for a period of three years, much to the dismay of his peers in the government, the number of women dying in various hospitals went up — not because there were more deaths, but because the previous numbers were inaccurate.
Over the last decade, Ethiopia has made major strides in health. There is an awful lot that still needs to happen, but the trajectory looks promising. As I was hearing the story from the Minister the other day (that I verified later from other colleagues) here in Addis Ababa, I wondered about our own data. How good is our data and whether it can be trusted?
When I was working on my book on drug quality, I wanted to get data about the prevalence of substandard and counterfeit drugs. What I got instead were numbers that were absolutely bizarre. And these were all from government sources. A minister would one day argue that less than one per cent drugs in the country were substandard, and another day some other government official would increase that to about 50%. The DRAP data was far from complete and internal inconsistencies made it impossible to make any sense of it. Even today, there is no national level surveillance on drug quality across the board and on where the problem really lies.
In my recent work on antibiotic resistance, again, there is little data about the burden of resistance in Pakistan. A few hospitals (largely in Lahore and Karachi) are collecting some data, but city, provincial or national-level data is non-existent. Rural areas might as well be on another planet. Hospitals serving underprivileged communities, where the burden is likely to be more acute, are data-free zones. For my research, when I went to some public hospitals where the burden of patients is high, there were no incentives to collect data and if it was collected, it was with archaic tools, likely prohibiting any further analysis.
Some colleagues would rightly argue that there is data on nutrition and statistics on infant and maternal mortality. That is true — but there are two problems with that argument. First, often this data is collected with partnerships and financial support from international NGOs — not as a routine practice. Second — national level policy cannot be made with data from just a few sectors. The policy that comes out of data-free scenarios is impulsive and misguided.
The obvious question is why is it that we don’t have quality data? There can be many secondary answers — from finances, to its political implications, prioritising other things and limited capacity. But the primary answer is simple — we simply do not care to collect it. That problem is no one else’s fault. It is of our own making.
Published in The Express Tribune, October 29th, 2019.
There was no incentive to be honest and truthful. Aman’s controversial policies incentivised accurate reporting, and for a period of three years, much to the dismay of his peers in the government, the number of women dying in various hospitals went up — not because there were more deaths, but because the previous numbers were inaccurate.
Over the last decade, Ethiopia has made major strides in health. There is an awful lot that still needs to happen, but the trajectory looks promising. As I was hearing the story from the Minister the other day (that I verified later from other colleagues) here in Addis Ababa, I wondered about our own data. How good is our data and whether it can be trusted?
When I was working on my book on drug quality, I wanted to get data about the prevalence of substandard and counterfeit drugs. What I got instead were numbers that were absolutely bizarre. And these were all from government sources. A minister would one day argue that less than one per cent drugs in the country were substandard, and another day some other government official would increase that to about 50%. The DRAP data was far from complete and internal inconsistencies made it impossible to make any sense of it. Even today, there is no national level surveillance on drug quality across the board and on where the problem really lies.
In my recent work on antibiotic resistance, again, there is little data about the burden of resistance in Pakistan. A few hospitals (largely in Lahore and Karachi) are collecting some data, but city, provincial or national-level data is non-existent. Rural areas might as well be on another planet. Hospitals serving underprivileged communities, where the burden is likely to be more acute, are data-free zones. For my research, when I went to some public hospitals where the burden of patients is high, there were no incentives to collect data and if it was collected, it was with archaic tools, likely prohibiting any further analysis.
Some colleagues would rightly argue that there is data on nutrition and statistics on infant and maternal mortality. That is true — but there are two problems with that argument. First, often this data is collected with partnerships and financial support from international NGOs — not as a routine practice. Second — national level policy cannot be made with data from just a few sectors. The policy that comes out of data-free scenarios is impulsive and misguided.
The obvious question is why is it that we don’t have quality data? There can be many secondary answers — from finances, to its political implications, prioritising other things and limited capacity. But the primary answer is simple — we simply do not care to collect it. That problem is no one else’s fault. It is of our own making.
Published in The Express Tribune, October 29th, 2019.