Better Medicare Alliance Unveils New Research on Limitations of MA-PD CAHPS Survey, Outlines Recommendations for Modernization

Washington, D.C. – Better Medicare Alliance’s Center for Innovation in Medicare Advantage (CIMA) today unveiled new research conducted by NORC at the University of Chicago examining the current limitations of measuring patient experience in the Medicare Advantage and Prescription Drug (MA-PD) Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey.

The 68-question MA-PD CAHPS survey is sent by mail annually to a random sample of beneficiaries in MA-PD plans. Since 2012, beneficiaries’ responses to this survey have been used to inform Medicare Advantage plans’ Star Ratings – the federal government’s system for measuring health plans’ quality on a 1 to 5 scale. Plans that receive the highest Star Ratings receive quality bonus payments used to enhance care and benefits.

NORC surveyed 800 Medicare Advantage beneficiaries and interviewed 41 experts across 20 organizations representing a cross-section of health care stakeholders to identify challenges and specific changes that would address these shortcomings.

“At Better Medicare Alliance, we fully support holding Medicare Advantage accountable for beneficiaries’ experiences and support gathering data through consumer surveys to enable plans and providers to maintain high consumer satisfaction,” said Allyson Y. Schwartz, President and CEO of the Better Medicare Alliance. “But, as this research from NORC at the University of Chicago shows, there is work to be done to make the MA-PD CAHPS survey more meaningful, actionable, and accurate. Response rates have fallen by more than 37% in the last decade as beneficiaries cannot access the survey online and are bogged down by the length and number of extraneous questions that are outside their Medicare Advantage plan’s control. With so much resting on this assessment tool, we owe it to beneficiaries to get it right.”

Schwartz continued, “The research from NORC not only identifies the problems but also offers sensible recommendations to modernize this measurement tool. By bringing the MA-PD CAHPS survey into the 21^st century with a web component, condensing survey questions to get to the heart of consumer experiences that are within MA plans’ control, and providing more geographic-specific survey data to plans in order to spur continued improvement, we can do right by Medicare Advantage beneficiaries, providers, and plans alike.”

“There are opportunities for the MA-PD CAHPS survey to modernize and better reflect how health care is delivered today. Now more than ever, consumer-centric care relies on teams of providers and may be delivered fully or partially via telemedicine, neither of which are captured within the survey,” said Caroline Pearson, Senior Vice President of NORC at the University of Chicago. “For health plans, it is essential that consumer experience metrics capture their diverse membership and the varied needs of beneficiaries, particularly as these measures are weighted more heavily.”  

Read the full report HERE and a fact sheet HERE. A brief summary of the report is provided below. 

Limitations of the MA-PD CAHPS Survey

• The MA-PD CAHPS survey does not capture the diversity of Medicare Advantage beneficiaries and does not address what matters most to beneficiaries in selecting a health plan or how beneficiaries experience care today.
  • For example, survey questions do not use more current health care terminology or reflect recent evolution in benefit design and care delivery, such as telemedicine, social determinants of health, non-physician health professionals, and health care delivered via integrated care teams.
• Health plans do not currently receive granular survey data that would provide information the health plans could utilize to improve the consumer experience.
  • The current MA-PD CAHPS survey results are aggregated at the contract-level and results are not provider or geographic specific. Consequently, there is little actionable data to drive quality improvement.
• A good number of MA-PD CAHPS survey domains are not within health plans’ control.
  • Numerous measures are completely or largely controlled by the provider rather than the health plan, including office wait times and following up on results. NORC’s survey of Medicare Advantage beneficiaries found that 88 percent of respondents agree their provider – rather than their health plan – is responsible for in-office wait times. Responses show a similar view as to doctor’s office follow-up, appointment wait times, and reminders for prescription refills. In addition, these questions are duplicative of surveys on consumer satisfaction with providers and could be eliminated in CAHPS.
• Limitations of data quality and consumer response rates.
  • In 2019, the MA-PD CAHPS survey saw an all-time low response rate of 38.4% compared to 61.7% in 2010 – a drop of more than 37%. Length of the questionnaire and utilization of only a paper-mode survey are a few reasons for the decrease in responses. A survey of Medicare Advantage beneficiaries found three out of four respondents (76%) preferred to receive and complete a survey via web/email. Falling response rates without additional information about data quality post a future reliability problem for the MA-PD CAHPS survey. 

Policy Recommendations

• Modernize patient experience measurement.
  • Include questions that speak to the care coordination, care management, and navigation roles payers increasingly play for their MA members.
  • Include questions that ask about patient experience with virtual appointments and visits, which may include ease of scheduling the virtual visit, timely follow-up from the provider and/or plan on the virtual visit, visits with non-physician health professionals, and overall satisfaction with the virtual visit.
  • Modify questions to capture health IT advancements that payers and providers use to communicate test results and coordinate follow-up care.
  • Consider additional user testing to address and account for the ways that individuals from different cultural backgrounds perceive care and how they respond to standardized survey questions.
  • Include questions that capture the factors beneficiaries say are most important to them when choosing a plan, such as affordability and the provider network.
• Provide more granular survey results to health plans while protecting beneficiary confidentiality to empower better health plan quality improvement.
  • Empower the MA-PD CAHPS survey vendors, or designate another third-party, to provide de-identified CAHPS results that provide more actionable information to the plans. This should include more information about providers, respondent demographics, and geographies, but not beneficiary identifiable information.
• Remove MA-PD CAHPS questions that health plans cannot directly impact.
  • The qualitative research highlighted that many questions on the MA-PD CAHPS survey ask about experiences that are outside the control of health plans. Similarly, beneficiary survey responses show a similar pattern of attributing responsibility for things asked about in the survey to entities other than the MA plan.
• Explore ways to reduce the burden on beneficiary survey respondents in order to improve response rates and publish data quality metrics.
  • Conduct a pilot program that adds a web-mode MA-PD CAHPS option to assess the impact on completion rates.
  • Conduct a transparent exercise with MA plans and survey methods experts to test the validity and reliability of reducing the number of MA-PD CAHPS questions.
• CMS should not increase the weighting of patient experience measures in the Star Ratings System until after modernizing patient experience measurement.
  • Pause the MA Stars patient experience weight increase set to take effect in 2023.

###