Estimating Deaths From Cardiovascular Disease: A Review of Global Methodologies of Mortality Measurement

Introduction

According to the National Center for Health Statistics, of 2 471 984 total deaths in the United States in 2008, 616 828 were caused by heart disease and 134 148 were caused by cerebrovascular disease (Table 1). By these statistics, heart disease is the leading cause of death in the United States.¹ Worldwide, the Global Burden of Disease study estimated that in 2001, 12.45 million of >56 million total worldwide deaths were caused by cardiovascular disease (CVD) and cerebrovascular disease. Ischemic heart disease was the leading cause of global mortality, accounting for 1.4 million deaths in the developed world and 5.7 million deaths in developing regions.²

These impressive statistics are used for the design and subsequent evaluation of health policies and interventions, and increasingly, attention is being drawn to the need to redistribute funding on the basis of disease and mortality burden.³ But how much do practitioners know about the strength of data such as these? Many cardiovascular clinicians and researchers consider mortality data to be the most basic type of data on which to make interventions and policy decisions. Although the conclusions drawn from various clinical trials may be challenged, it is generally taken for granted that CVD mortality data in developed countries are valid. But mortality data, like all data, are subject to limitations based on how the data are collected. Furthermore, as the cardiology community begins to focus its attention on the prevalence of CVD in developing countries, it is important to determine how best to assess CVD-associated mortality in settings where most people die at home without death certificates or prior health records. The purpose of this review is to elucidate how mortality statistics are currently collected in both developed and developing countries, the various limitations of such data collection in each of these settings, and some strategies for improving how we currently measure CVD mortality (Figure 1).

• Download figure
• Download PowerPoint

CVD Mortality Measurement in Developed Countries

In the United States, the National Vital Statistics System uses death certificates to collect mortality data under the jurisdiction of each state.⁴ Diagnoses on the death certificates are then converted to a diagnostic code from the International Classification of Diseases (ICD), which is the standardized classification of diagnoses developed by the World Health Organization (WHO). Vital record systems using death certificates are also used in other developed countries such as the United Kingdom, Sweden, and Italy.

Inaccuracies of Death Certificate Diagnoses

In reality, death certificate diagnoses are often inaccurate as a result of either erroneous clinical diagnoses or incorrect or inadequate coding of the diagnosis.⁵ This has been a long-standing problem. In 1955, James et al⁶ found that in a comparison of 1889 death certificates with corresponding autopsy reports, only 52% of cases had full agreement between 3-digit ICD codes. A 1980 series of 257 autopsied cases found that 42% of cases had improper recording of the underlying cause of death.⁷ Malignant neoplasms were underreported by 10%, and vascular diseases were overreported by 10%.

More recently, Roulson et al⁸ performed a meta-analysis of articles published between 1980 and 2004 that studied discrepancies between clinical and postmortem diagnoses. The authors found that the rate of misdiagnosis has not significantly improved since the 1960s: At least one third of death certificates are likely to be incorrect, and 50% of autopsies produce findings unsuspected before death, underlining the importance of autopsy. These studies show that the inconsistencies and inaccuracies of cause-of-death reporting have been an issue over the past several decades and continue to present a major problem.

It can be argued that autopsy discrepancy studies such as those outlined above are biased because the autopsied cases may have been those in which the cause of death was particularly obscure clinically. However, in a 1980 study by Cameron et al,⁹ an autopsy rate of 65% was reached for 6 months to try to diminish the effect of selection bias. They found that in 38% of cases the cause of death was still discrepant between the autopsy findings and the death certificate, even when clinicians were “certain” or “fairly certain” about their diagnoses. Interestingly, in cases in which clinicians indicated that they would not normally have requested an autopsy, the discrepancy rate was similar to those cases in which they would have. This finding indicates that it is difficult for a clinician to predict which cases will have discrepant diagnoses.

Furthermore, a 1992 study in East Germany by Modelmog et al¹⁰ autopsied 96.5% of deaths in a 1-year time period in the city of Goerlitz and compared these results with the death certificates. The authors found that 47% of diagnoses on the death certificate differed from that on autopsy and that, in 30% of all subjects, these differences crossed a major disease category. Interestingly, this study confirmed that diseases of the circulatory system were likely to be overdiagnosed. These results are similar to those of autopsy studies, which do not randomly sample patients or include all subjects, indicating that selection bias alone is unlikely to account for their results.

Inaccuracies in death certificate coding are not unique to the United States. A study of death certificate accuracy in Sweden during 1995, in which death certificate diagnoses were compared with causes of death gleaned from case summaries of prior hospitalizations, found that only 77% of death certificate diagnoses were considered accurate.¹¹ Of note, these authors found that heart disease other than ischemic heart disease was particularly prone to erroneous death certification.

Inaccuracies of Cardiovascular Deaths on Death Certificates

As alluded to above, deaths resulting from CVDs in particular appear to be difficult to assess on death certificates. Ravakhah¹² performed a study of 223 autopsy cases in 2006 and determined that 48% of autopsy-proven cases of acute myocardial infarction were misdiagnosed as something else (ie, cases of underdiagnosis) and that 25% of cases certified as acute myocardial infarction were actually erroneous (ie, cases of overdiagnosis). Interestingly, 21.5% of the death certificates studied were of little value because of inadequate diagnosis such as “cardiopulmonary arrest,” “arrhythmia,” or “respiratory failure.” This issue of using ill-defined cardiovascular codes on death certificates was examined by Lozano et al,¹³ who found that in certain countries the rates of death resulting from ischemic heart disease may need to be corrected by up to 30% to adjust for the use of these ill-defined codes. The authors argue that the increases in death rates from ischemic heart disease over the last 20 years in countries such as Austria, Japan, and Spain may be due at least in part to improvements in coding practices.

Whether the result of miscoding or other causes of misclassification, several studies have shown that overdiagnosis of deaths resulting from cardiovascular causes is a problem in certain developed countries. In Japan, Hasuo et al¹⁴ showed that in a comparison of the death certificate and cause of death assigned by autopsy, the false-positive rate of assigning CVD as the underlying cause of death was 22%. Similarly, the UK Heart Attack Study Collaborative Group showed that in 3 districts of the United Kingdom between 1994 and 1995, death certificates assigned CVD as the cause of death in people >65 years of age 20% more often than would have been assigned by strict clinical and pathological criteria.¹⁵

Using the Framingham Heart Study in the US, Lloyd-Jones et al¹⁶ tried to determine the accuracy of death certificates for coding coronary heart disease as the underlying cause of death. They used as their gold standard a cause of death adjudicated by a panel of 3 physicians and found that the death certificate assigned 24.3% more deaths to coronary heart disease than did the physician panel. In decedents >85 years of age, more than twice as many deaths were assigned to coronary heart disease. These data would indicate that CVD may be overrepresented as a cause of death on death certificates and thus that national mortality statistics may significantly overestimate CVD as a cause of death. Clearly, using a physician panel that was removed from the case at the time of death as a gold standard has its limitations. However, the authors contend that the similarity of their findings with those of other studies argues against significant inaccuracy in the physician panel diagnoses.

Within the category of deaths resulting from CVD, sudden cardiac death poses particular difficulty because physicians have to determine the cause of death outside the hospital. Several studies have looked at this issue specifically. Fox et al¹⁷ looked at out-of-hospital coronary heart disease on death certificates compared with physician-adjudicated sudden cardiac deaths in the Framingham Heart Study from 1950 to 1999. Sudden cardiac death was defined as a coronary heart disease death that occurred within 1 hour of symptom onset and was without other probable cause of death suggested by the medical record. A panel of 3 physicians used hospital and primary care records and next-of-kin interviews to determine probable cause of death. They found that out-of-hospital coronary heart disease deaths as reported on death certificates overestimated physician-adjudicated sudden cardiac death by 47% and that this overestimation was significantly increased in women and older patients. Other studies have confirmed an overestimation of out-of-hospital coronary heart disease death rates by death certificates.^18,19

Thus, even though there is a specialized system in place to classify causes of death in developed countries such as the United States, there remains a large degree of inaccuracy in the system with resultant uncertainty as to the exact mortality rate attributable to CVDs. Specifically, evidence suggests that the rate of death from CVDs is being overestimated with the reporting system currently in place. However, it is reassuring that despite this potential overestimation, cardiovascular disease mortality in the United States appears to be declining over time.²⁰

CVD Mortality Measurement in Developing Countries

Although death certificates clearly have their limitations, they are a rarity and relative luxury in many developing nations. Nonetheless, targets such as those set forth by the WHO to reduce deaths from noncommunicable diseases by 25% by 2025 rely on mortality data to assess whether they are met.²¹ Without reliable death documentation for the majority of deaths, how are developing countries addressing the issue of mortality measurement? If countries do not have complete death registration systems with medically certified cause-of-death information, they can use sample vital registration systems that draw on both death certificates and verbal autopsy studies to extrapolate mortality estimates. In the absence of this, they can resort to cause-of-death modeling.²²

Status of Vital Registration Systems Abroad

The majority of developing countries have no functioning vital registration system of any kind. In a comprehensive review of current global data on death registration, Mathers et al²³ found that of 194 member states, 115 reported mortality data to the WHO (Figure 2). Of these, only 23 had data on >90% of the population, used ICD codes to assign causes, and reported <10% of deaths as being due to ill-defined causes.²³ In Africa, 42 of 46 member states were unable to provide any mortality data from 1990 or later, and more than half of these (n=25) were unable to provide any mortality data at all. Consequently, data are provided to the WHO annually for ≈18.6 million deaths, which is estimated to be only one third of the deaths occurring in the world. These figures reflect the dire state of death reporting in many developing countries.

• Download figure
• Download PowerPoint

Of the mortality data provided by many developing countries, much of the information is fraught with problems. Such countries have to overcome an obvious obstacle, which is that the majority of people die at home without medical attention, making the exact cause of death difficult to determine.²⁴ In addition, economic, social, religious, and political factors can make reliable cause attribution at home difficult. It has been shown that in some countries with high HIV burden, certifiers can be pressured by family members not to mention AIDS on the death certificate.²⁵ Finally, the same errors that diminish death certificate accuracy in developed countries are likely at play in developing countries as well, including clinical misdiagnosis and miscoding.

Sample Vital Registration Systems

One alternative to complete vital registration systems, which are costly and require significant infrastructure that often does not exist, is to develop a sample vital registration system, as has been done in India and China. In India, a sample of nationally representative households is being monitored between 1998 and 2014 in an effort to assess the vital status and cause of death of nearly 14 million individuals.²⁶ Cause of death will be determined from verbal autopsy methods, and then national mortality estimates will be extrapolated from these sample data. Although there are several pitfalls to using verbal autopsy data, as discussed in the following section, this project is still expected to provide substantially better mortality data than was available previously. This study has already yielded national estimates of tobacco-related deaths,²⁷ cancer mortality,²⁸ child and adult mortality resulting from unintentional falls,²⁹ infectious disease mortality in children,³⁰ child mortality resulting from unintentional injuries,³¹ snakebite mortality,³² mortality associated with malaria,³³ and overall causes of neonatal and child mortality in India.³⁴

The sample vital registration system in China, or the Disease Surveillance Point System, is similar in that a representative 1% of the population is used to determine cause-specific mortality statistics through death certification and verbal autopsy methods.³⁵ Although additional studies to validate the results from this effort are necessary, a 2007 study suggests that many of the diagnostic misclassifications that occur in urban death registration data appear to balance each other out at the population level, but caution is required when drawing conclusions about specific causes of death in China.³⁶

Verbal Autopsy

The verbal autopsy method can be used to collect primary data on a patient’s likely cause of death by interviewing the primary caregivers about the signs, symptoms, and circumstances that preceded the death. Generally, a questionnaire is administered by an interviewer to a family member who was most familiar with the circumstances surrounding the decedent’s death. The data from this questionnaire are then examined either by multiple physicians or by a computer algorithm program that then assigns a likely cause of death. verbal autopsy makes the assumptions that most causes of death have distinct symptom complexes that can be recognized and recalled by lay observers and that it is possible to classify deaths into useful categories of causes of death on the basis of reported information.³⁷

There are several limitations to the verbal autopsy methodology of obtaining mortality data. The first is that verbal autopsy protocols have not been standardized across studies. In the early 1990s, the WHO recommended key design features for verbal autopsy studies.³⁸ These included the following 7 recommendations: The questionnaire should have both structured and unstructured questions; interviewers should be specially trained; interviewees should have been in close proximity to the deceased during illness; only a short time interval should have elapsed between death and data collection; algorithms for translating the data into causes of death should be clearly defined; there should be a possibility of assigning multiple causes of death; and subsequent validation studies should be carried out.

However, these guidelines do not appear to have had much impact on the designs of subsequent verbal autopsy studies. Joshi et al³⁸ conducted a systematic literature search of all verbal autopsy studies published before January 2006 and compared the study designs both before and after the above guidelines were issued. There was no clear increase in the proportion of studies that used the 7 key indicators as outlined above after the guidelines were published. The authors concluded that the recommendations for optimal verbal autopsy study design have been implemented incompletely to date. They note, however, that the reason could be that the guidelines were based mostly on consensus expert opinion rather than on scientific evidence.

A similar outcome was found in a 2006 comparison of verbal autopsy protocols in 36 field sites that determined that the methodologies and standards used varied substantially across sites.³⁹ This lack of standardization makes it essentially impossible to compare data across communities and countries, which severely limits the usefulness of these data.⁴⁰ In another effort to facilitate standardization of verbal autopsy protocols, the WHO has developed several resources for verbal autopsy studies, including standardized verbal autopsy questionnaires for 3 age groups.⁴¹ It remains to be seen whether these resources will prompt better compliance with standardized protocols.

A second major limitation to verbal autopsy studies is the inherent difficulty in validating the results. Ideally, autopsy studies would be used as the gold standard to which verbal autopsy diagnoses would be compared. However, thus far, it appears that only 1 study has been published that uses autopsy to validate verbal autopsy results, and that study focused specifically on deaths from acute respiratory infections in children in Mexico City.⁴² It showed that symptoms such as cough and dyspnea used in verbal autopsy had fairly good sensitivity (0.61 and 0.69, respectively) and specificity (0.73 and 0.74, respectively) in predicting death resulting from acute respiratory infection in children compared with autopsy. It should be noted, however, that in many places autopsy may not be culturally acceptable, leading to strong selection bias if the selected cases are atypical.³⁷ In addition, autopsy studies may be prohibitively expensive in many developing nations.

Instead of using autopsy as the gold standard, verbal autopsy validation studies often use hospital diagnoses for those patients who were recently seen in a hospital setting. However, this can also introduce selection bias because patients who use hospitals may have different access to care, different treatment success, and different socioeconomic characteristics compared with those who do not.^37,43 In addition, the reliability of hospital diagnoses themselves is likely to be at least as questionable as in developed countries, if not more so. Nonetheless, in the absence of better alternatives, several verbal autopsy validation studies using hospital diagnoses and physician review of medical records as the gold standard have been performed. Chandramohan et al⁴⁴ explored the validity of a verbal autopsy tool in Tanzania, Ethiopia, and Ghana compared with hospital diagnoses. They found that their tool was particularly sensitive for acute febrile illness, direct maternal causes, tuberculosis/AIDS, tetanus, rabies, and injuries. In a validation study of verbal autopsy against hospital diagnoses in Thailand, Polprasert et al⁴⁵ found that their tool had a 49.8% sensitivity and 50.8% positive predictive value for ischemic heart disease, leading to a cause-specific mortality estimate of deaths outside the hospital caused by ischemic heart disease of 5.9% in men and 6.5% in women.

Verbal autopsy validation studies need to be interpreted with caution because nearly all published validation studies have used information from the household on death certificates, medical records, and importantly, on recall of cause of death assigned by a healthcare worker.⁴⁶ This would likely bias the verbal autopsy validation results so that they match the medical record–derived diagnoses. Another caveat of validation studies is that ideally they need to be performed in every population in which the verbal autopsy tool is being used because the sensitivity and specificity of the tool depend on underlying cause-specific mortality fractions that vary by population.^47,48

Recognizing the inherent problems associated with using hospital diagnoses as the gold standard in verbal autopsy validation studies, Murray et al,⁴⁹ as part of the Population Health Metrics Research Consortium, set out to provide a true gold standard data set to use to validate verbal autopsy tools. This data set consisted of >12 000 hospitalized patients in 4 countries for whom stringent diagnostic criteria, including laboratory, pathology, and medical imaging findings, were used to determine gold standard causes of death. A verbal autopsy instrument based on WHO standards was then performed on all gold standard cases. Using this gold standard data set, Lozano et al⁵⁰ showed that using verbal autopsy data, physicians assigned the gold standard cause of death <50% of the time, which is far less robust than previously thought.

Epidemiological Modeling

If minimal or no mortality data exist, as is the case in many low-income countries, 1 way to overcome this obstacle is to use epidemiological modeling to fill in the gaps left by patchy or nonexistent data, as was done in the 1990 Global Burden of Disease Study.⁵¹ This effort was important because it attempted to predict the cause-of-death structure in areas where there is otherwise no information on how many people are dying and their causes of death. Updated estimates of all-cause mortality were made for 2008; for countries without nationally representative mortality data, these estimates were derived from projections based on 2004 estimates, as well as fragmented data from registries, verbal autopsy studies, disease surveillance systems, and analyses from WHO technical programs.⁵² Currently underway is the Global Burden of Disease 2005 study, an update of the 1990 study, for which new epidemiological methods are being developed to account for incomplete data.⁵³ Similar to the 2008 update, even fragmentary sources will be used to determine condition-specific mortality estimates, and it is expected that several iterations of estimates will be necessary before all possible sources of data are incorporated accurately. It remains to be seen whether this method of epidemiological modeling will be sufficiently accurate in determining mortality in regions with minimal data.

It is important to note that the predicted estimates are only as reliable as the underlying data, which, as discussed above, are often inaccurate to begin with.²⁶ Burden-of-disease estimates can vary considerably, depending on the assumptions used. For example, modeling estimated that in India there were 0.78 million cancer deaths in 1990; however, cancer registry data suggested a much lower figure of 0.43 million deaths.⁵⁴ This suggests that there is no substitute for primary data when measuring mortality and determining cause of death structure.

Strategies for Improved Mortality Measurement

CVD mortality statistics are often assumed to be correct by cardiology clinicians and researchers alike. However, as the above discussion shows, the reliability of CVD mortality data in both the domestic and international settings has much room for improvement (Table 2).

In developed countries, 1 method of improving the quality of mortality data would be to systematically teach physicians how to fill out death certificates properly. The fact that so many diagnoses on death certificates are “garbage diagnoses” such as “cardiac arrest” or “cardiovascular collapse,” which are of course the result in all deaths, speaks to the need to train physicians in documenting underlying diagnoses. Indeed, very little formal training exists for physicians on this issue. Rather, most physicians learn to fill out these forms from informal advice from medical records departments, peers, and even funeral directors.⁵⁵

In 1996, Messite and Stellman⁵⁵ published a small but informative study that demonstrates the need for physician training in death certification completion. Using 6 clinical cases prepared by the National Center for Health Statistics to instruct physicians on death certification, this study assessed whether 12 practicing internists, 21 internal medicine residents, and 35 senior medical students could correctly complete a death certificate. They found that the level of agreement between the underlying cause of death reported by the participants and the correct cause was 55.7% to 56.9% across groups. Although this was only a small pilot study, it demonstrates the continued need for physician training in death certificate completion and underscores the need for formal instruction in medical school and residency programs. Several studies have evaluated various educational interventions to improve death certificate accuracy^56,57; although most of these interventions lead to improvement in accuracy, the most effective interventions consist of interactive workshops rather than printed material alone.^58,59

Even if physicians are better trained at providing cause of death diagnoses clinically, there still needs to be an increased frequency of validation of these diagnoses with autopsy. With regard to CVD deaths in particular, autopsy improves the overall accuracy of CVD statistics because it can both determine when patients have died of CVD causes and distinguish definitively between different types of CVD deaths.⁶⁰ Unfortunately, autopsy rates in the United States have been decreasing progressively: The rate in 1958 was 19.1% and in 2007 was only 8.5%^61,62 (Figure 3). This is likely due at least in part to cost because the medical examiner’s office often does not reimburse the full cost of autopsy. Plus, the fear of malpractice litigation and the belief that with modern diagnostics, the clinical cause of death is certain with modern diagnostics has likely also played a role.⁶⁰ In addition, the decision of the Joint Commission on Accreditation of Hospitals to drop a 20% to 25% autopsy rate standard for hospital accreditation in 1971 likely also contributed.⁶¹ The standard was dropped with the idea that hospitals in various situations should not be expected to conform to the same standards with regard to autopsy. However, a more nuanced standard could potentially increase national autopsy rates and should be considered.

• Download figure
• Download PowerPoint

In developing countries, implementing large-scale sample vital registration systems using verbal autopsy, as is currently being done in India, will likely make great strides in reliably estimating cause-specific mortality for a significant portion of the global population.²⁶ Standardization of verbal autopsy protocols according to guidelines, along with validation of the guidelines themselves, would greatly improve the quality of the mortality data that these studies provide. In addition, validation studies of verbal autopsy with necropsy should be encouraged. To the best of our knowledge, no verbal autopsy validation studies with necropsy have been performed for adult deaths. As necropsy studies in the United States have shown, there can be great discrepancy between death certificate diagnoses and actual causes of death, and necropsy studies are the only way to truly prove the validity of a verbal autopsy tool.

Conclusions

Clinicians and researchers alike should recognize the limitations of even the most basic data on which decisions are being made. The cardiology community must look at mortality data with a critical eye, as with all other types of data, but must also work within the constraints placed by minimal resources in developing countries to make as accurate an estimate as possible. Improving physician training in filling out death certificates and encouraging autopsies will better the situation in developed countries. In developing nations, implementing sample vital registration systems using verbal autopsy tools, standardizing verbal autopsy protocols, and using necropsy to validate verbal autopsy tools will lead to improved mortality data collection.

Footnotes