The rationale, design, and implementation of the American Cancer Society's studies of cancer survivors

Institutional Review Boards, State Registry,and Survivor Eligibility Criteria

Overall approval for the SCS was obtained from the Institutional Review Board (IRB) of Emory University. Additional IRB and regulatory approvals were obtained for each registry. In both studies, state cancer registries were selected based on the following considerations: 1) to maximize coverage of the United States population, targeting states with large, diverse populations representative of the different regions of the country, with at least 1 state from each census region (West, Midwest, Northeast, and South); 2) to maximize coverage of ACS divisions, attempting to target at least 1 state per division; 3) to include state registries capable of implementing the SCS recruitment protocol; 4) to include state registries where data were available to draw a sample of eligible cancer survivors. SCS-I survivors were sampled from 10 months to 11 months after diagnosis and, thus, must have been ascertained and recorded in the registry database by that time. SCS-II included survivors 10 years after diagnosis; thus, registries must have contained data on survivors this long postdiagnosis at the time of sample selection. Table 1 lists the 11 states that were included in SCS-I and the 14 states that were included in SCS-II. Four states participated in both studies. Survivors who were included in the studies met the eligibility criteria provided in Table 2.

Sampling

A separate, state-wide probability sample was drawn from the central cancer registry in each state in which either study was conducted. The exceptions were California, where separate probability samples were drawn for Regions 2, 3, 4, 5, and 6, and Washington State, where the sample was drawn from 13 counties. In both studies, the sample was stratified by cancer type. Racial stratification was not conducted within a state if the distribution of survivors by cancer type, age/cohort, and race/ethnicity lead to empty or sparse cells, because this would prohibit the calculation of meaningful parameter estimates. In contrast to other cancers, skin melanoma generally was not stratified by race/ethnicity because of the lower incidence rates and absolute numbers of this cancer among minorities.61 Table 1 lists the racial strata used in each state. In states where the sample was stratified by race/ethnicity, minorities were over sampled up to the point of achieving equal allocation across racial/ethnic groups. The SCS-I sample was stratified by age group, with survivors aged <55 years over sampled to allow adequate numbers of younger survivors for analysis. In SCS-I, the survival rates for each cancer were incorporated into the sampling strategy to ensure a sufficient sample at 2 years, 5 years, and 10 years postdiagnosis. The cancers with the best 1-year and 5-year survival rates comprised the first group (melanoma, breast, prostate, bladder and uterine), the cancers with moderate survival comprised the second group (colorectal, kidney, NHL, and ovarian), and lung cancers comprised the third group. For every 1 patient selected in the first group, 1.4 survivors were selected from the second group, and 3 survivors were selected from the lung cancer group. The selection fraction for lung cancer was a compromise that balanced needs for precise estimates of QOL in 2-year, 5-year, and 10-year lung cancer survivors; the expected number of lung cancer patients available for sampling; and overall study costs.

In SCS-II, the general design allocated the state sample size equally across each of the postdiagnosis cohorts. Within each cohort, 25% of the sample size was allocated to each of breast cancer, colorectal cancer, and prostate cancer; and 12.5% of the sample was allocated to each of the remaining cancers (melanoma, bladder cancer, and uterine cancer). This allocation strategy reflected the distribution of these 6 cancers in the United States, with more of the sample allocated to the more common cancers, producing an approximate probability proportional to size-stratified sampling plan across cancer types.

State cancer registries are required to report data 2 years after the close of a given calendar year. Because SCS-II targeted survivors ≥2 years postdiagnosis, nearly all survivors who ultimately were identified by a registry had been ascertained. SCS-I, however, targeted survivors 1 year postdiagnosis, so samples were drawn before state cancer registries had completed case ascertainment and data cleaning. Consequently, the database from which SCS-I samples were drawn did not include all eligible survivors. Efforts to increase coverage of eligible survivors in SCS-I included drawing a series of subsamples to increase time for case ascertainment and, in 1 state, the use of rapid case ascertainment. The state-level databases from which SCS-I samples were drawn contained a median of 63.2% of the survivors who ultimately were identified by the cancer registry (range, 30.9–118.5%); numbers >100% resulted from duplicate records, that is, multiple records from different facilities for a single case. Duplicates were removed after sampling.

Questionnaire Development, Pilot Testing,and Translation

The SCS surveys include instruments and scales that are used widely in psychosocial research and that are known to be valid and reliable for use with cancer patients (Table 3). In addition, for areas of interest with no established instruments, items were developed and pilot tested. Questionnaire development involved 3 major activities: 1) consultation with a panel of medical and behavioral cancer researchers, 2) administration of the questionnaire to focus groups, and 3) pilot testing the questionnaires and study protocols with samples of survivors selected from registries. The final versions of both surveys took approximately 60 minutes to complete. The questionnaires and all other study materials were translated into Spanish by a panel of experts, except where validated Spanish versions of scales already exist. These translations subsequently were tested with Spanish-speaking focus groups.

Physician and Survivor Consent

Recruitment procedures varied, depending on the rules and regulations regarding recruitment in a given registry. Physicians were identified through registry records and follow-back to reporting facilities. Physicians either were contacted for written consent to recruit a given survivor (active physician consent), were sent a letter explaining that recruitment of a given survivor would begin in 3 weeks unless study personnel heard from the physician (physician notification), or were informed about the study in general through mailings and presentations to medical societies and hospitals. This last approach did not involve the identification of individual survivors. Table 1 identifies the recruitment procedures used by each registry.

Survivor recruitment also varied based on registry regulations. Twelve registries released survivors to the ACS for simultaneous consenting and surveying, whereas 16 registries consented survivors prior to releasing contact information to the ACS for survey administration. One registry simultaneously consented and administered the baseline survey. With regard to recruitment, all registries used an adaptation of Dillman's tailored design method,62 which includes 3 mailings—1) a cover letter, consent form, and/or questionnaire, 2) a reminder letter, and 3) another consent form and/or questionnaire with a different cover letter—followed by telephone follow-up.

Eligibility and Response of Sampled Cases

Although the designs of the 2 studies differ, the protocols and recruitment rates are similar. The final dispositions of all sampled survivors for both studies are presented in Figure 1. For the combined studies, 61,847 survivors were sampled, of which 9.1% were determined to be ineligible for recruitment after sampling because of missing or out-of-range values in eligibility variables, such as stage or date of birth. This left 90.9% eligible for physician consent. During recruitment, survivors were determined to be ineligible or deceased based on information obtained from the individual or a proxy (eg, medical professional or family member). Survivors were coded as unlocatable when mail was returned undeliverable, the phone number was not working, or someone indicated that the individual did not reside at that address, and additional contact information was not found despite extensive efforts. Response rates were estimated as the number of completed interviews divided by the number of eligible reporting units in the sample63; ineligible survivors were removed from the denominator. Because large portions of survivors were identified as deceased during recruitment despite efforts to update vital status before and after sampling, they were reported separately from other ineligible survivors. The physician consent rate (PCR₁) of 91.9% is the proportion of survivors still eligible at this stage—excluding those ineligible or deceased at physician consent—that received physician consent and became eligible for case consent. The case recruitment rate (CRR₁) of 35.1% is the proportion of participants eligible for case consent—excluding those deemed ineligible or deceased—that completed a survey. Finally, the overall recruitment rate (ORR₁), 32.0%, takes into account the combined impact of physician and case consent by indicating the proportion of eligible survivors who completed a survey.

The ORR₁, however, underestimates the response rate, because it assumes that everyone with unknown eligibility—that is, survivors coded no response or unlocatable—are eligible refusals. In accordance with standard survey procedures,63 response rates were recomputed after the number with unknown eligibility was adjusted downward to account for the fraction that would most likely have been ineligible. It was assumed that the fraction of ineligible survivors for the unknown eligibility group was the same as the fraction for the known eligibility group. An ineligibility fraction was calculated for each recruitment stage, and the number of survivors with unknown eligibility that were assumed ineligible was computed and removed from the response rate calculation. For example, the ineligibility fraction at the physician-consent stage is the number of survivors identified as ineligible or deceased divided by the number of survivors identified as eligible for case consent, physician refused, ineligible, or deceased. The ineligibility fraction is then applied to the survivors with unknown eligibility—no response and unlocatable—to estimate the number of ineligible survivors to be removed from the response rate calculation. Adjusting the SCS rates by using this approach results in minor changes to the PCR₂ (92.0%), CRR₂ (37.3%), and ORR₂ (34.0%).

Response by Study Site and Cohort

The PCR₂, ORR₂, and numbers of completed surveys are shown for each site for both studies in Table 1, which also provides the type of physician consent and survivor recruitment used by each site. Response rate calculations reflect differences in site recruitment procedures. The sites that required active physician consent (for SCS-I, Connecticut and Ohio; for SCS-II, Colorado and Nebraska) have a much lower median PCR₂ (62.0%) than sites where physician notification was used (median, 96.5%). The median ORR₂ across registries is 34.9% (minimum, 15.9%; maximum, 52.1%).

Table 4 presents the distribution of survivors in final disposition for each cohort: 1 year for SCS-I and 2 years, 5 years, and 10 years for SCS-II. The 1-year cohort had higher levels of deceased and ineligible survivors than later cohorts. One trend is the increasing proportion of unlocatable survivors, which rises from 4.8% in the 1-year cohort to 10.1% in the 10-year cohort. Similarly, the proportion of survivors lost to physician consent increased from 5.9% in the 1-year cohort to 8.5% in the 10-year cohort. It also is worth noting that the 10-year cohort had a lesser proportion of survivors completing questionnaires. To rule out the possibility that differences between cohorts were caused by differences in study sites, the same comparisons were made using only data from states that recruited all cohorts, and the patterns noted here remained unchanged.

Representativeness and Descriptionof the Final Sample

Calculated by using registry data, the ORR₁ for particular medical and demographic groups is presented in Table 5. For both studies, survivors who were younger (18–54 years), women, and white responded at higher rates than survivors who were older (≥55 years), men, and racial/ethnic minorities. Ovarian and breast cancer survivors had the highest response rates, and bladder and lung cancer survivors had the lowest. Survivors who were diagnosed initially with distant or in situ cancer responded at lower rates than survivors who were diagnosed initially with local or regional cancer. The same pattern was observed when these comparisons were repeated with data from the 4 states that participated in both SCS-I and SCS-II.

Among the respondents, 19.4% are racial/ethnic minorities (nonwhite). Based on self-reported data, 40.1% of respondents have a high school education or less, and 16.2% have an annual household income <$20,000. In SCS-I, baseline surveys were completed a median of 1.2 years after diagnosis. Respondents in the 2-year cohort of SCS-II completed surveys a median of 3.3 years after diagnosis, respondents in the 5-year cohort completed surveys a median of 6.3 years after diagnosis, and respondents in the 10-year cohort completed surveys a median of 11.3 years after diagnosis.

Sampling and Recruitment

Studies of cancer etiology must survey patients who die soon after diagnosis because of the possible relation between high-level exposure and death shortly after diagnosis, and this requires the use of costly rapid case ascertainment (frequent visits to reporting facilities to ascertain cases). In contrast, studies of long-term outcomes do not require data on patients who die shortly after diagnosis and can utilize more economic approaches. Studies of outcomes ≥2 years past diagnosis can rely on inherent ascertainment activities, because registries are required to report to their regulatory agencies—the Surveillance, Epidemiology, and End Results Program; the National Program of Cancer Registries; and the North American Association of Central Cancer Registries—at that time. When drawing samples closer to diagnosis, the cost of ascertainment can be contained by using 2-stage cluster samples to decrease the number of facilities to be visited and by making visits less frequent. For example, studies of 1-year survivors need visits to facilities only 1 or 2 times per year. Finally, the sampling design of registry-based studies should take into account the data presented in Table 4. The increasing proportion of survivors who are unlocatable or lost to physician consent in cohorts further from diagnosis probably resulted from deterioration of registry data, whereas higher rates of ineligible and deceased survivors in the 1-year cohort probably were due to sampling from an incomplete registry database.

Although the physician consent process provides data on eligibility and vital status, it increases costs and decreases response rates. Active physician consent becomes problematic as survivors move beyond initial treatment and stop seeing the physicians recorded in the registry. The SCS states that required active physician consent had lower physician consent rates (median, 62.0%) than the rate achieved by those using physician notification (96.5%), which requires fewer resources, is quicker, and places fewer burdens on health professionals. Furthermore, the majority of patients (87%) do not believe that a physician should decide whether they are approached for a study.69 Based on SCS data, state cancer registry personnel from 3 states successfully changed regulations that required active physician consent. Even physician notification can be problematic if registries do not collect physician data and if reporting facilities refuse to provide physician data despite registry IRB approval. Because of issues with physician data, 1 IRB approved informing physicians about the study through large physician groups and state medical associations rather than physician notification. In summary, physician notification is preferable to active physician consent in survivor studies; the quality of physician data is important; and regulatory bodies are amenable to changing their policy on physician notification if the evidence is persuasive.

It is known that survey salience has an impact on survey response,70 and a number of factors may have decreased the salience of SCS to survivors. Some SCS participants indicated that cancer was no longer important to them, some preferred not to be reminded of it, and others believed that their stories were uninteresting, because they had been cancer free for years; recruitment materials were developed to address these issues. The SCS surveys were conducted on a statewide basis, with recruitment materials generally featuring the ACS and the state department of health; in contrast, surveys that are identified as coming from universities, particularly local universities, usually get better response.62 Compared with single cancer studies, the salience of the SCS may have been diluted further, because the multiple cancers included did not allow survey materials to match a participant's diagnosis.

The receipt of study materials was the first time most survivors became aware that a state agency had their name, address, and information about their cancer. Many were shocked by this, which may have overshadowed consideration of study participation. For those who were contacted by telephone, explanations of the registry's objectives and the steps taken to protect patient privacy often overcame participant concerns. Sending a prenotification letter about the cancer registry before study materials are sent may attenuate the potential impact of patients learning that they are listed in a cancer registry. This would be consistent with Dillman's tailored design method,62 which posits that providing information in more digestible chunks leads to better survey response. Furthermore, prenotification increases response rates by 3% to 6% in the general populace,62 and evidence suggests that prenotification may increase participation in registry-based surveys.69

When designing registry-based studies, researchers should consider issues of ascertainment, sampling, and recruitment as well as techniques like face-to-face interviews, incentives, or community-based research to improve response rates. Because of their cancer experience and the discomfort that may be associated with the discovery of being listed in a cancer registry, recruits for registry-based studies are a unique group who may respond differently to survey design elements than the general public. The field of registry-based surveys would benefit from the integration of methodological experiments into protocols in an effort to identify improved recruitment approaches.

Probability sampling from state cancer registries has the advantage of providing full coverage of the population of survivors in a defined geographic region. The SCS show that it is practical to conduct multisite studies of the QOL of long-term cancer survivors using state cancer registries. The only comparable studies are 2 National Cancer Institute-sponsored studies that were designed to investigate the relation between cancer care practices and outcomes, including survival and QOL.71, 72 The National Cancer Institute-sponsored studies sample from state cancer registries and abstract data from medical records but collect less psychosocial data and include survivors of fewer cancers than SCS. Therefore, the inferences regarding long-term survivorship that can be drawn from the SCS are the most wide-ranging to date.