Taking a Page from Education to Improve Healthcare

Education has borrowed many ideas from the medical field, ranging from residency programs for principals-in-training to classroom rounds for teacher preparation. Now a new initiative shows the exchange isn’t just a one-way street.

Bookmarking—a widely-used method for establishing student proficiency levels in major education tests such as the National Assessment of Educational Progress (NAEP)—is being adapted to healthcare so patients and their families can better communicate the severity of symptoms to their medical providers. AIR was recently recognized for this innovation by an award from the National Quality Forum, a leader in establishing standards for healthcare quality and measurement, as part of its Innovation Challenge.

In education, bookmarking combines statistical analyses with expert input to divide student assessment scores into categories reflecting different levels of achievement. Bookmarks are cut points—specific scores on an assessment that represent the dividing lines between achievement levels such as the NAEP categories of basic, proficient, and advanced achievement.

Here is why bookmarking can be a benefit to doctors: To provide the best care, healthcare providers need to understand their patients’ symptoms and how those symptoms are affecting their patients’ lives.

Quantifying patients’ experiences with symptoms and functioning through patient-reported assessments—what the healthcare world calls “patient-reported outcome measures”—can guide better care decisions in hospitals or doctors’ offices. Measured outcomes include symptoms such as pain, fatigue, and depression, as well as levels of physical functioning such as the ability to climb stairs easily.

Measuring these outcomes directly from the patient’s perspective can help researchers determine what treatments work best for particular conditions. Patient-reported outcome measures could also be used by insurance companies, employers, or policymakers to understand which hospitals, doctors, or clinics provide the best care to their patients.

The key to implementing patient-reported outcome measures into practice is properly interpreting assessment scores. What does a score of 8.2 on a patient-reported outcome measure of pain mean for a patient battling arthritis? Is that severe pain?

If that patient’s score later drops to 6.4 after starting a new treatment, is that a low level of pain? Is that a meaningful change in the pain this patient is experiencing?

Describing symptoms in categories such as mild, moderate, or severe can be more useful than using a numeric score by itself, but how do you decide which scores represent mild, moderate, or severe symptoms? And who decides?

That’s where bookmarking comes in.

In an ongoing study funded by the National Institutes for Health, researchers at AIR are adapting bookmarking to translate scores on patient-reported outcome measures into meaningful categories for healthcare providers, patients, and their families. Through a series of bookmarking workshops, AIR experts are soliciting input from adolescents with cancer or arthritis and involving their parents in developing bookmarks that will divide the score scale into mild, moderate, and severe categories. The series of workshops—conducted separately with each group—also includes physicians who specialize in treating pediatric cancer or arthritis.

During the workshops, each group will first discuss definitions for the categories of “none,” “mild,” “moderate,” and “severe” symptoms and of “no problems,” “mild problems,” “moderate problems,” and “severe problems” related to functional status. It is important for participants to agree on concise descriptors as they will influence the precision of the bookmarking results.

Once descriptors are chosen for these categories, AIR will walk each group through the process of placing bookmarks between vignettes that describe different levels of symptoms and functionality experienced by hypothetical patients and how that affects their daily lives. As they place bookmarks between sets of vignettes that represent more or less severe experiences, workshop participants will be setting cut points, dividing patient-reported outcome scores into those categories.

The physician group also will indicate treatment interventions associated with each level of severity or functional status.

The end goal of these bookmarking workshops: developing symptom and functional categories that accurately reflect what adolescent patients with arthritis or cancer experience—bookmarks that are useful and easily interpretable by the people caring for them.

AIR is currently developing the workshops, so results are still months away. In the meantime, with help from the National Quality Forum, AIR is sharing the approach and lessons learned along the way with others in healthcare, beginning with a webinar on December 14.

Michelle Langer is a senior psychometrician AIR. She specializes in using statistical methods for all types of tests, including measures of educational achievement and patient-reported outcomes.

Ellen Schultz is a senior researcher with AIR. Her work focuses on patient and family engagement, improving the quality of healthcare, and finding ways to measure quality of care.