Here’s a cautionary tale.
Two years ago, you may have heard that Texas’ maternal mortality rate had just doubled. No one was quite sure why—some blamed a recent shuttering of family-planning services, others a lack of concern for women’s health generally—but it seemed most were confident Texas was a cautionary tale. The U.S. maternal mortality rate overall had been rising for years, even as nearly every other developed country saw declines. Surely, the new numbers out of Texas portended what could happen across the country if we didn’t change soon.
It turns out that the numbers in Texas were wrong. The extent to which they were incorrect was just published in Obstetrics & Gynecology, as investigated by several researchers who went back and double-checked the cause-of-death for 147 total deaths in Texas in 2012.
Though we only now realize how flawed the data were, this isn’t entirely a recent revelation. Experts suspected that something was wrong with the data, they just weren’t sure what. The authors of that original study noted that “in the absence of war, natural disaster, or severe economic upheaval, the doubling of a mortality rate within a 2-year period in a state with almost 400,000 annual births seems unlikely.” Their suspicions turned out to be right. Instead of roughly 36 deaths per 100,000 births, the mortality rate was more like 14.6 deaths per 100,000 births.
Texas is, in fact, a cautionary tale, just not in the way we all thought. It’s been collecting data poorly for years now, and they’re not alone—maternal mortality rates could be wrong all over the U.S. Ever since some states introduced a checkbox to their death certificates asking about pregnancy back in 2003, the stats on maternal mortality have been skewed. A 2017 study concluded that this addition “appears to be the main driver of the increases in [maternal mortality rates] during the last decade.” States with and without the checkbox differ so much in reported data that we haven’t published an official national maternal mortality rate since 2007. The data just haven’t been good enough.
But they have to be better. Maternal mortality rates are still high nationally by many estimates, and understanding the extent of the problem is crucial if America is going to save mothers’ lives.
When researchers found that twofold increase in maternal mortality in Texas two years ago, they suspected it was so extreme as to be an error. Texas was one of the states that implemented a new checkbox on death certificates that asked about the pregnancy status of the deceased, and as mentioned before it was already known that states with that checkbox tend to report higher maternal death rates. This is in part because asking about whether the deceased was pregnant increases the rate at which it’s reported—when you start looking for pregnancy-related deaths, you’ll tend to find more. But it’s also because people make mistakes.
The recent paper notes that it may be a simple design problem that’s contributed significantly to the apparent rise in maternal mortality: “Texas’ current electronic death registration system displays pregnancy status options as a dropdown list. The “pregnant at the time of death” option is directly below the “not pregnant within the past year” option; this could have led to erroneous selection and could explain why pregnancy at the time of death was reported for nearly 76% of the 74 obstetric-coded deaths with no evidence of pregnancy on review.”
The authors also note that the number of death certificates being submitted electronically in Texas jumped from 63 to 91 percent from 2010 to 2012. Having a bunch of new users entering information into death certificate software may have exacerbated the misreporting.
They suggest two main solutions: better training for workers who report death information, and instead of a drop-down menu, try buttons. Separate buttons for “pregnant” and “not pregnant” force the user to move the mouse to totally different areas to make a selection, rather than clicking in a small drop-down menu where it’s easy to make a mistake and not realize it before moving on to the next question.
Texas may have had a double-whammy situation, but the problems they ran into are true of all checkbox states.
Those states that chose to implement a pregnancy checkbox on their death certificates have seen a 149.8 percent in maternal mortality (as of 2012 data). But a study from last year estimated that a whopping 90.3 percent of that change was likely due to the checkbox issues alone.
“Because pregnancy-related deaths are so uncommon, the frequency of the box being checked in error can significantly impact the maternal mortality rate reported,” Elliott Main, medical director of the California Maternal Quality Care Collaborative, told the Washington Post.
If we assumed that the non-checkbox-adopting states showed the true rate, the study authors reasoned, the true rate would be 14.4 percent.
Lesson: Bad UI design kills! Or at least fools you into thinking someone’s been killed…
(Hat tip: Instapundit.)