Science | ASCQ-Me

A Consumers Working Group was created to meet and explore these issues from 2002 to 2003. Among the influences to quality of life that emerged from the working groups were: 

• Cost of living with sickle cell disease;
• Individual, family, and community resources/support systems that are needed; 
• Religion, spirituality, prayer, institutions, and the faith community;
• Dating, relationships, and sexuality;
• Family, parenting, and marriage;
• Satisfaction with treatment and care;
• Access to care;
• Burden of treatment;
• Emotional health (e.g. self-esteem, depression);
• Educational attainment;
• Employment opportunities;
• Individual coping strategies; and
• Discrimination.

With input from the Survey Research Working Group, these topics were grouped into the following domains:

• Somatic sensations, including symptom burden, pain, fatigue, and a general sense of well-being;
• Physical role function;
• Social role function, including family, intimate, and peer relations;
• Psychological functioning, including emotional and cognitive function;
• Spirituality; and
• Patient and provider/health care system interactions.

With the exception of patient and provider/health care system interactions, the Survey Research Working Group reclassified the consumer-generated topics into those that are often included in health-related quality of life (HRQOL) measures.

The research team used the results of the NHLBI working groups to inform the selection of key words for the literature review, as well as the development of protocols for the patient focus groups. The team then integrated what was learned from the NHLBI Working Groups with what emerged from the literature, and from adults with SCD through the focus groups, to produce a draft conceptual framework. This framework was modified and expanded based on the results from the patient and provider critical incident interviews.

Articles were included if they contained self-report of the experiences of adults with Sickle Cell Disease (SCD) and were also included if they were about community or outside perceptions of SCD. Articles were excluded if they did not include self-report of the lived/felt experience of adults with SCD and/or health-related quality of life; or where treatment, rather than patients’ symptoms and functioning, was the focus of the article.

Literature Analysis

Our analysis of the literature revealed that many of the life effects uncovered in the literature review were those typically included in measures of health-related quality of life (HRQOL): pain, fatigue, emotional distress, social relationships, ability to carry out daily activities as well as social roles (worker, student, parent, etc.). 

Other classes of life effects were typical of topics covered in patient health care experience surveys such as access to treatment and interactions with providers, specific side-effects or discomfort associated with treatments, and methods of coping with and managing a chronic condition. 

Many of the effects of SCD uncovered by the literature review had to do with the way individuals feel or perceive themselves in relation to society including perceptions of stigmatization, feelings of isolation and alienation, and consequent shame surrounding disclosure of SCD status.

Participant Recruitment

Focus group participants were adults with SCD representing a diversity of demographics and geographic locations. Participants were recruited using flyers describing the study that were distributed to sites providing clinical care for adults with SCD; through member organizations of the Sickle Cell Disease Association of America (SCDAA); and through other SCD community organizations.

Procedures

The focus groups were facilitated discussions led by moderators (nurses, psychologists, and social workers) who had experience working with individuals with SCD. The groups were homogeneous regarding gender and moderators were matched to the gender of the group. The purpose of gender matching was to facilitate candid discussion about gender-related influences of SCD on a person’s life. The majority of the focus group protocol involved structured discussion to elicit critical incidents, i.e., the effect of SCD on the participants' lives. 

After each focus group generated its own critical incidents, participants were asked to rate the importance of aspects of health-related quality of life identified by the consumer working groups and the literature review. To do this, participants were asked to review the list of health-related quality of life aspects and vote on which aspects were most important. Participants were given seven votes and they were allowed to give any aspect more than one vote. Participants were also asked to vote on their preferences for the modality with which a health-related quality of life measure might be administered (e.g., paper and pencil; computer; telephone).

Results

Characteristics of participants. Eighty-four adults participated in 11 focus group interviews (39 male in five groups, 45 females in six groups). Adults with SCD ranged from 18 to 64 years of age (median 35 – 44 years) and represented the range of SCD genotypes. Consistent with the population of individuals affected with SCD in the U.S., 89% of focus group participants were Black or African American. Focus groups were conducted in urban, suburban, and rural sites in Arkansas, California, Florida, Maryland, Massachusetts and New York. Focus groups were hosted by community-based organizations, hospitals, and a university. 

Prioritization of SCD life effects. As expected, pain received the most votes, followed by fatigue/tiredness, work, and physical activities. Relationships with family and friends were voted on relatively infrequently, as were volunteer work issues—each of these categories received less than two percent of votes. Sexuality issues also received only two percent of votes even though the topic generated quite a bit of discussion in the groups. The pattern of votes cast by men and women was similar across domains, with the exception that women chose issues of tiredness and fatigue significantly more often than men (12% versus four percent, chi square (1df) = 9.98, p = .002) and men chose emotional issues (8% versus four percent, chi square (1df) = 5.00, p = .025) significantly more often than women. 

Survey methodology. Participants responded to a series of questions about previous survey experiences and preferences for how a measure of health-related quality of life (HRQOL) for adults with SCD might be administered. The most frequently preferred mode of administration was a computer with a keyboard (34%), followed by paper-and-pencil administration (28%), and a telephone interview conducted by a live person (23%). Participants were in favor of the development of the HRQOL measurement system, but very much not in favor of giving out personal information if it is not going to be listened to, or not getting feedback about the results of the project.

Qualitative Notes from Groups

After logistics and ground rules were described by the moderator, and group members introduced themselves, the participants were asked the question: When I say the words “sickle cell” what do you think of? The overwhelming response was “pain,” followed by a variety of words that focused on the disease and its treatments: “blood disease,” “hospital,” “transfusion” and “medication.” Other descriptors focused on the impact of the SCD: “burden,” “torture,” “hardship,” “discomfort” and “limitations.” A few group members commented on potential positive impacts: “balance your life” and “self-aware.” There was very little need to probe for critical incidents, as the focus group members were very forthcoming regarding positive and negative effects that SCD had had on their lives. 

Following the critical incident portion of the focus group, participants were asked: How would your life be different if you didn’t have sickle cell? Both men and women commented on limitations they would not have if they did not have SCD. This included the ability to travel more; participate in sports; pursue a different educational or career path; and to live wherever they would like, without regard to the weather or being away from family. Men often commented on their limitation regarding employment in the military or law enforcement. Some participants focused on not having to have contact with the health care system if they did not have SCD. Not all participants felt their lives would be different in any way. 

In response to What bothers you most about having sickle cell? Participants focused on its unpredictable course, costs of treatments and medications, and negative judgments in relation to their need for pain medications. Some commented on disappointing their families and friends due to illness episodes, missing out on events and holidays, and on their children seeing them ill or having to help them when they were ill. 

Participants were asked If you did not have sickle cell, what would you miss? Participants had many responses to this question, including positive traits they might not have developed—patience, persistence, humility, compassion, and empathy. They commented on missing the connections and friendships they developed with other people with SCD as well as with health care providers. 

As happens in focus groups, group members responded to topics that emerged, and for the most part, validated one another’s ideas. For example, men and women talked about concerns with under-employment, limited income, and medical expenses. Concerns that individuals with SCD seeking pain medication could be/are often stereotyped as addicts frequently emerged. The topic of health promotion was validated. Health promotion activities mentioned included eating healthy, drinking lots of water, and learning how to deal with stress or emotional situations. Race issues included: greater perceived empathy from ethnic minority providers, lack of information in the African American community itself, and lack of attention and under-funding because SCD is perceived as a “Black” disease.

The CIIs were conducted during the same period of time as the focus groups. We sought to interview both adults with SCD and their providers, with the idea that the provider interviews would supplement information that was provided by patients. As with the focus groups, patients were recruited using flyers describing the study that were distributed to health care sites, and through SCD community agencies.

Recruitment and Data Gathering

Participants who were recruited for the interviews to represent a broad range of experience with SCD including both genders, a span of ages, and SCD genotypes. Interviewees were recruited from urban and rural areas in 22 states. Similarly, a variety of types of health care providers were recruited.

All 15 clinicians and 36 persons with SCD were interviewed individually by telephone. The remaining 84 persons providing critical incidents participated in the in-person focus groups. Individuals with SCD were asked to think about times that SCD had either positive or negative effects on their lives. Clinicians were asked analogous questions about individuals with SCD that they knew.

The CIIs were a highly efficient method of qualitative data collection: the 138 persons interviewed provided 1,213 critical incidents for analysis. We created the incidents by listening to tapes of the interviews and following a prescribed format for transforming the anecdotes to text.

A senior team member reviewed each incident to insure that the standards for an adequate critical incident were met: (1) The incident described a SINGLE behavior, (2) The behavior was CRITICAL to the outcome (positive or negative impact on quality of life due to SCD), (3) Another person could understand what is going on, (4) Non-essential information was eliminated, and (5) unstated inferences did not have to be made. Incidents not meeting these criteria were either sent back to the interviewer for correction or were eliminated. Sixteen incidents (1.3%) were eliminated. 

Data Analysis 

Qualitative data are analyzed by identifying themes or similarities among the materials provided by multiple interviews. The CIT uses a process of taxonomy development to identify themes. The CIT also uses independent analysis followed by discussion to consensus to ensure the stability and generality of the themes. Thus, the project team was then divided into two teams of two members each which independently categorized the same 200 incidents by identifying the key behavior responsible for the positive (or negative) outcome and briefly describing this behavior. 

The two teams met to compare their preliminary set of categories and to derive a single, agreed- upon taxonomy for the incidents. Discussions centered around the areas of difference. Solutions included convincing the other team to change their interpretations, or developing more general categories that incorporated differences or breaking down categories that were too large to accommodate the detail perceived by the other team. 

Additional sets of approximately 200 incidents were then categorized. The teams developed specific and higher-order categories as needed to accommodate incidents that were deemed by all to be qualitatively different from the rest. 

Finally, the last group of incidents resulted in the creation of only three single-incident categories within the higher-order structure; that is, no unique general themes emerged indicating saturation. The resulting 140 specific categories of incidents had been grouped into 16 summary categories : emotions; attitudes, beliefs and behaviors; family and social relationships; morbidities and co-morbidities; sexuality and reproduction; medical care; economic issues; employment issues; education; faith/spirituality; altruism; stress (control and predictability); activity limitations; housing issues; community issues; and idiosyncratic responses.

Content Validity

In this section we describe our approach to developing Sickle Cell Disease (SCD)-specific content, developing items based on patient input, complementing the content of the Patient-Reported Measurement Information System (PROMIS) item banks, and determining the appropriate recall period.

Attribution to Condition 

Two basic approaches typically are taken to develop any condition- or disease-specific patient-reported outcome (PRO) or health-related quality of life (HRQOL) measure. These are methods based on: 

1. Condition attribution, wherein the question is considered disease-specific because it attributes the symptom or deficit in functioning to a particular condition or disease. 
2. Content validity, wherein the question is considered disease-specific because it was generated from one-on-one or group interviews with people who have the condition. 

ASCQ-Me is a measure of how adults with SCD feel and function, so we had to decide whether and how to build attribution to SCD into the questions. We took the second approach to generating questions for ASCQ-Me. That is, we did not create SCD-specific questions by adding the phrase “because of my SCD”, rather, we based the content of questions on numerous in-depth interviews with adults living with SCD. Read more about why here.

Patient Input 

In the previous section (Critical Incident Interviews), we described the critical incident technique (CIT) and how it contributed to the conceptual framework that the ASCQ-Me research team generated. This section demonstrates how the critical incidents were used to write items for ASCQ-Me. 

More than 1,213 incidents were generated from the CIT. From this collection, 830 questions were written. The example below illustrates how questions were generated from the CIT.

Question for the emotional impact item bank: 

Critical incident 

"I had 5 close friends..[pass].. from sickle cell..it is hard for me to enjoy life sometimes because I am constantly trying to plan when my life is going to end.” 

Question 

In the past 7 days, how often did you feel that SCD could take your life at any moment? 

Answer 

Never –to– Always 

Question for the stiffness item bank: 

Critical incident 

"I wake up in the morning and feel like I’m 80 yrs. old. I am stiff. My back and joints will feel out of place. I have to spend a lot more time in the morning just to get started." 

Question 

In the past 7 days, how often were your joints very stiff when you woke up? 

Answer 

Never –to– Always 

Complementing PROMIS Content 

The 830 items written from the CIT were reviewed for redundancy, and a reduced list was compared to the questions in PROMIS item banks that targeted similar health concepts: PROMIS Anger/Aggression, Anxiety, Depression, Fatigue Experience, Fatigue Impact, Pain Quality, Pain Interference, Physical Functioning, Social Role Performance, and Sleep/Wake Functioning. The goal of this process was to determine whether ASCQ-Me had enough unique content to warrant developing an item bank. Items eliminated from the ASCQ-Me item set were items that were nearly identical to PROMIS items. In fact, the entire group of ASCQ-Me items targeting Fatigue and Pain Interference were redundant with those in the PROMIS item banks.

Item banks. Five item banks were developed for ASCQ-Me to cover emotional aspects of living with SCD (Emotional Impact), pain particular to SCD (Pain Impact), degree of stiffness and interference caused by stiffness (Stiffness Impact), interference with sleep characteristic of SCD (Sleep Impact), and particular ways that SCD interferes with or shapes social activities (Social Impact). At the time that we developed the ASCQ-Me questions, there was no finished PROMIS measure for cognitive functioning so we developed questions for that domain of health as well, but we did not find enough evidence of validity for the ASCQ-Me cognitive functioning item bank to recommend it for use. Read more about our work on assessing cognitive functioning. In the meantime, PROMIS had developed two item banks for cognitive functioning. We recommend that these be tested in adults with SCD. 

SCD severity items. We required a method of identifying groups of patients that might differ in their degree of disease severity so that we could evaluate that ability of ASCQ-Me measures to describe the impact of variations in SCD severity on the feeling and functioning of adults. The ASCQ-Me project team consulted with providers who treat patients with SCD to generate a list of questions that providers might ask in evaluating disease severity. These questions included six conditions (different types of organ damage, bone damage, and leg ulcers) and two treatments (daily pain medication and regular transfusions). 

Pain episode items. Five questions asking about the frequency, severity, and duration of pain due to vaso-occlusive events were drafted based on how these events were described in the CII’s. These questions are meant to be an indicator of severity in addition to those listed in the SCD severity medical history checklist. 

Quality of care items. ASCQ-Me includes a fixed-format questionnaire, called the ASCQ-Me Quality of Care measure, to understand the health care experience of adults with SCD. This measure is not integrated into the ASCQ-Me domain framework for health because patient reports of the quality of their care are theoretically distinct from PRO or HRQOL measures in that they are descriptions of care processes, not outcomes. Donabedian’s often-cited model to support health-services research includes three categories of variables: structure, process, and outcome. Process measures include patient interactions with providers and what was done to the patient and when. Outcome measures include indicators of patient health following the delivery of health care. Patient reports can provide data for both health care process and outcome measures. The Consumer Assessments of Healthcare Providers and Systems (CAHPS) family of surveys is an example of measures based on patient reports of health care delivery process. PROMIS is an example of measures based on patient reports of their health. The ASCQ-Me QOC measure is based on patient reports of health care delivery process. PROMIS is an example of measures based on patient reports of their health.

Appropriate Recall Period 

Traditionally, PRO or HRQOL questionnaires ask patients to respond to a time frame that would capture the state of well-being or functioning that is currently most characteristic of respondents and not unduly influenced by rare, acute symptoms. For this reason, a recall period that is as short as possible while being resistant to the influence of an immediate, but fleeting and uncharacteristic health event is desired. For example, the SF-36 contains many questions that ask the respondent to refer to the “past 4 weeks” (20 of 36 questions). But other SF-36 questions ask the respondent to refer to the unspecified present (15 of 36). And one question asks respondents to compare their health now to their health one year ago. By contrast, PROMIS has chosen the “past 7 days” as the most common reference period for questions. However, some PROMIS domains use the unspecified present as well or a period of 30 days. 

In developing the time frame of questions for ASCQ-Me, we had the following objectives:

1. ASCQ-Me should use the shortest time frame possible to describe the person’s state of being that is currently characteristic of him or her and that is appropriate to the response scale;
2. Once decided, ASCQ-Me should not change the question reference period from one administration to another because such a change would make comparisons among administrations impossible (i.e., one would not know whether a difference in scores was caused by difference in the time frame of the questions or to the difference of interest); and
3. Finally, because ASCQ-Me was designed to complement PROMIS, we sought to adopt a time frame that is consistent with the PROMIS, as long as it is also appropriate to the ASCQ-Me content.

With input from the advisory committee of adults with SCD, guidelines for recall periods were drafted for question writers to follow: 

The unspecified present should be used for items that refer to characteristic ways of thinking or behaving that are stable over time, such as: 

• Ways of coping with SCD—coping mechanisms, disease management.
• General self-perceptions—self image.
• General perceptions of how others view the self—discrimination/stigmatization.

The 7-day period should be used for experiences that have the potential to occur several times a day or week: 

• Pain—this can occur several times a day or more than once in a 7-day period.
• Sleeping problems—we sleep (or try to sleep) at least 7 days a week. 
• Fatigue—we can experience fatigue several times a day or more than once in a 7-day period. 
• Difficulty thinking, reasoning, paying attention—we use these faculties several times a day and certainly more than once in a 7-day period.
• Depression, anxiety, anger—it is possible to have these emotions several times a day or several days out of 7.
• Problems performing at paid work, household work, or in school. 

The 30-day time frame would be appropriate for social activities that are not certain to occur once a week: 

• Engaging in recreational activities with friends.
• Engaging in recreational activities with family.
• Sexual behavior.

A 12-month time frame would be appropriate for activities that are not certain to occur once a month: 

• Engaging in community activities.
• Career-related activities.
• Changes in social networks.
• Periods of absence from school—like career related.
• Getting behind in school or work (due to absence)—like career related.
• Hospitalizations, transfusions, other health care–related events.

Evaluation of Question Comprehensibility 

The director of AIR’s Cognitive Testing Lab reviewed all of the candidate ASCQ-Me items and identified those which had the slightest potential to be misunderstood. These were included in the cognitive testing protocol. Two rounds of cognitive testing using the highly sensitive “think-aloud protocol" were conducted. The first round covered the entire set of items recommended for testing and the second round covered the changes that were made following the first round. In each round, the items to be tested were divided into two halves. One half was tested with six respondents and the other with five respondents, for a total of 11 respondents per round. Each round used a formal cognitive interview protocol to probe comprehension and navigation. Four trained cognitive interviewers conducted the 22 interviews on two coasts: North Carolina and California. All interviews were audio recorded and the notes from the interviews and audio-recordings were analyzed.

Results 

Sickle cell disease severity items. The demographics section of the questionnaire included disease-severity items, which characterized the current status of disease progression among respondents. They determined whether the respondent had ever been diagnosed with a particular condition, in contrast to the HRQOL items, which assessed recent or current symptoms and ability to function. Many participants had trouble understanding the terms used to refer to different conditions or diagnoses (e.g., avascular necrosis). In most instances, this issue was addressed by providing lay descriptions of the different conditions followed by the technical name of the condition (in parentheses). For example, “Has a doctor or nurse ever told you that you have leg ulcers (open sores on your legs and feet)?” was rewritten as: “Have you ever had open sores on your legs or feet (leg ulcers)?” 

Pain episode. The term “pain episode” was misunderstood by most respondents. When asked, “When was your last pain episode?” some interpreted it too narrowly, as only referring to instances when hospitalized, while others interpreted it too broadly, as referring to all types of pain, including pain that was mild or chronic. We used the term “pain episode” rather than “pain crisis” or “sickle cell crisis” in deference to many advocates who feel that “crisis” should not be used because it connotes an overwhelming and extremely dangerous situation, potentially reinforcing feelings of helplessness and hopelessness and preventing people from developing proactive coping strategies. The project advisory board was polled regarding its preferences for an alternative term, and a compromise solution was developed in which the phrase “…pain episode (crisis)” was used to refer to the sudden onset of severely debilitating pain. 

Willingness to respond. All respondents were positively disposed toward completing the questionnaire and indicated that, if asked by their physician, they would complete the instrument. Respondents stated that a questionnaire like this might increase knowledge about SCD and could help others with the disease. At least two respondents noted that the questionnaire allowed them to tell people what it is like to live with SCD. 

Health as the reason for limitations. For some items about limitations, it was determined that it is necessary to specify health as the reason for the limitation. For example, when asked about needing help doing household chores, one respondent indicated that help was needed because the house was too big for any one person to clean, not because of health or SCD. To attribute a need for help to health rather than an environmental contingency, the phrase “because of your health” was added to the end of items about role limitation, including household chores, grocery shopping, and going to appointments.

Types of Assessments

Two general types of assessments were developed: 1) item banks, which enabled computer adaptive testing (CAT) and 2) short, fixed-format questionnaires, which could be administered by computer or by paper and pencil. 

Item Banks. The ASCQ-Me research team designed questions to target different levels of the continuum underlying functioning or severity of symptoms (e.g., from impossible to move to easy to move). This is consistent with the use of the item response theory modeling approach taken by PROMIS. Using this approach, items for emotional impact, physical impact (including stiffness, sleep functioning, and pain), and social functioning were developed. 

Short, Fixed-Format Questionnaires. AIR developed fixed-format questionnaires for severity of SCD, pain episodes, and quality of care.

Site Selection

Seven field test sites from across the United States participated in ASCQ-Me. Sites were chosen based on their geographic diversity and variation in patient population characteristics. These locations included two community-based organizations, three academic medical centers, and two clinical sites. Each field test site was required to have:

1. A substantial population of adults with sickle cell disease (to allow the recruitment of a minimum number of participants) 
2. Institutional Review Board (IRB) approval for participation. AIR’s IRB served as the governing IRB for sites without their own IRB. 
3. At least three computers with audio and access to high-speed internet service, which were available to respondents for three-hour increments. 

Each field test site assigned a site coordinator, who was responsible for IRB submissions at that site, recruitment of participants, and assisting each participant in completing the survey. All site coordinators were trained in study procedures by members of the research team to ensure uniform data quality. The training covered participant recruitment, informed consent, and the ASCQ-Me web survey protocol and instructions. 

Participant Recruitment 

Two approaches were used for participant recruitment. First, the sites used existing clinical relationships with SCD patients in the area and recruited these patients to take the survey at their clinic or location. The second approach was a snowball recruitment technique: at the completion of the survey, patients were asked to let others with sickle cell disease (SCD) know about the project and ask them to contact the clinic if they would like to participate. 

Data Collection Procedure 

When participants arrived to complete the survey, site coordinators explained the process and asked them to sign a consent form prior to beginning the survey. The field test of ASCQ-Me was conducted via a web-based survey hosted on an AIR web server. After the site coordinator logged into the ASCQ-Me website, a unique respondent ID was created for the participant. The site coordinator helped the respondent to log in to ASCQ-Me and gave assistance where needed in completing the survey. 

Results 

The ASCQ-Me data analysis (specifically deriving item-level calibrations using a two-parameter graded response model) would require 500 completes and so the ASCQ-Me Research Team set its target total data collection for 550 to accommodate a ten-percent missing data rate. This was a very ambitious target because the adult SCD population is difficult to access, but the entire AIR research team worked hard to support the sites in exceeding this data collection target. This help included assisting with the data collection and recruiting new field test sites when the original sites were not able to meet target levels of recruitment.

Our purpose in using these methods was to understand the dimensional structure underlying the responses of patients to ASCQ-Me questions, including where the responses fell on the underlying health dimension tapped by each item bank and the amount of information each response and each question contributed to the dimension. The resulting statistics were used to create computerized adaptive tests (CATs). The most useful and informative items in each ASCQ-Me item bank were identified to create five-item, short forms which could be employed in place of the CATs for those users without access to computers.

Analysis of Subdomains and Question Scoring 

The ASCQ-Me SCD severity questions and the pain episode questions were not analyzed using IRT. Exploratory and confirmatory factor analyses were used to identify two subdomains within the pain episode question responses—frequency and severity. Then classical psychometric analyses were used to evaluate the construct validity and reliability of the two dimensions including internal consistency reliability for the subscales, and the magnitude of the item total correlations. 

We also compared the correlation of an item with its own scale (corrected for overlap) with its correlation to the other scale. We did not use structural methods to analyze responses to the SCD severity questions because we did not hypothesize that there should be any relationship between the responses to these questions. These were simply a checklist of medical conditions and treatments that are often associated with SCD but which could be independent of each other. 

Thus, we scored the ASCQ-Me SCD severity questions as an index using a method commonly employed for co-morbidity indices which is to create a score from the simple sum of the number of questions endorsed. We realize that this treats each question with equal weight and encourage others to conduct research using alternative weighting schemes for each question or, indeed, to use alternative questions.