Why is Monitoring Performance Important?

In late 2015, CMS announced a move from a fee-for-service model to a quality-based payment model.  The Accountable Care Act (ACA) contains some promising steps in that direction, although it’s future is questionable.  Even with that uncertainty, it looks like those laws will likely stay on the books for some time.  In California, they have dubbed their ACA exchange “Covered California: the California Health Benefit Exchange”.  Covered California has embedded quality of care deeply into is financial model requiring health plans to monitor and vary hospital contracting to avoid hospitals with poor quality measures around hospital-acquired infections and performance of C-sections which are not medically necessary1.  With these changes occurring, it is imperative that health plans understand their Star ratings and the key indicators.

CMS’s Medicare Stars encompass a lot of measures across many platforms with results from various timeframes.  HEDIS measures look back as much as ten years.  Consumer Assessments and Healthcare Providers Systems (CAHPS) surveys are conducted during the current measurement year, but are subject to members’ recollection around timing.  Health Outcomes Survey (HOS) results are based on comparison across multiple years and dependent on members perception of their quality of care.  In a nut shell, there are a mountain of underlying details that need to be understood and monitored.  To drive results, it is imperative to know where improvement is needed and then understand how to drive the needed improvement.

Each year, CMS publish a Star Ratings Fact Sheet that provides some insight, but nothing truly actionable.  The most clear-cut differentiation is non-profit Medicare Advantage plans with prescription drug coverage (MAPD) plans versus for-profit MAPD plans.  With the Stars 2017 results, the average non-profit MAPD plan scored an overall Star Rating of 4.0 Stars, while the average for-profit MAPD plan scored only 3.6 Stars.  Looking at the Prescription Drug Plans (PDP) with no Medicare Advantage coverage, the result is the same.  Non-profits averaged 4.0 Stars while for profits averaged 3.6 Stars.  When looking at the same data, but based on plan enrollment instead of the number of plans, MAPD non-profits averaged 4.3 Stars while for-profits averaged just 3.8 Stars. For PDP plans, the divide was even wider.  Non-profits average 4.3 Stars while for-profits average just 3.5 Stars.  While it’s not likely that an organization will switch either way, this does drive home the point members gravitate toward plans with higher star ratings, and members are premiums.


Another study included in the Star Ratings Fact Sheet is Overall Star Ratings by Length of Time in Programs.  Plans are divided into 3 groups: less than five years, five years to less than ten years, and ten years or more.  For the MAPD plans, the average Star rating for these three categories were 3.4 Stars, 3.6 Stars, and 3.9 Stars, respectively.  For the PDP plans, the average ratings were these: 3.6 Stars, 3.9 Stars, and 3.8 Stars, in each of the respective categories.  While one would expect that with time, plans would get better at managing these members over time.  The PDP plans with ten years or more time in the program did slightly poorer than those between five years and less than ten years.  While the MAPD plans matched the expectation of getting better with time.  Compounding the expectations that plans would improve over time is the fact that CMS discourages plans who are performing poorly to stay in business.  On April 12, 2016, CMS notified Cuatro LLC, an MAPD plan, of its decision to non-renew the company’s Medicare contract due to performance.  Cuatro failed to reach three Stars in for three consecutive years3.  Up to this point, such plans only received a low performing icon, , and sanctions, such as its members being allowed to move to other plans outside of open-enrollment.

While the contents of this paper are interesting, they are hardly actionable.  An organization is either for-profit or a non-profit and only time will move a plan through the three categories CMS includes in its years in programs.  Something more meaningful, more detailed needs to be put in place to provide clear visibility into possibilities and challenges.

Photo by {artist}/{collectionName} / Getty Images


There are several solutions that can be purchase purchased with varying degrees of cost, usefulness, and integration time.  An organization’s resources may be tied up for months getting the products set up.  There may be additional services needed to install the product, and the usefulness will certainly live up to the vendor’s expectations, but will it add value to the purchasing organization?  Here are some examples:

  • Inovalon has a Medicare Stars product called Star Advantage®.  Based on the description, this product requires a full integration with a plans member and provider data, indicating a pricey solution that will require a project with the plan’s IT team to arrive at the solution and subsequently integrate and implement the product.  From the description, it is unclear whether Inovalon’s Star Advantage® also requires the purchase of their EDIS Advantage™ product too.  It is unclear whether product includes CAHPS, HOS, Appeals, Interpretive Services, complaint tracking module (CTM), or adherence performance reporting.
  • Altegra Health has a solution called Star Program Analytics.  This platform is built on the same platform as their HEDIS Smart™, meaning that it is likely an additional purchase with the HEDIS Smart™ product.  Altegra offers provide and member-level drill down capabilities.  It is unclear whether product includes CAHPS, HOS, Appeals, Interpretive Services, CTM, or adherence performance reporting either.  Again, pricing information is not available on line, and an integration process is likely necessary.
  • McCormick BI, LLC offers a low-cost subscription cloud based solution that does not require any IT and project management resources for setup.  McCormick BI, LLC’s Stars Performance Analysis Tool Pack utilizes industry leading data visualization software called Tableau to deliver a robust interactive dashboards that allows plans to review their data relative to selected competitors.  It provides a one-screen overview that readily identifies areas of success along with areas of opportunity in an easy to share, easy to discuss layout.  It also allows for review of cut points, plan, and competitor performance over time at the measure or summary level.   

The McCormick BI, LLC Stars Performance Analysis Tool Pack, includes a suite of reports that provide various levels of information to health plans.  The first question most health plans are asking is, “How are we doing compared to our competitors?”.  The Analysis Tool Pack answers that question immediately.  The below screenshot is available right out of the box.  All that is needed is for the plan’s competitors to be identified.  That is only a matter of looking those competitors up by name or CMS contract number.  These options are demonstrated in Figure 1 to the right.  McCormick BI, LLC will also provide introductory training to understand the Stars Performance Analysis Tool Pack.  Even helping to identify your competitors, if needed.

Competitor Selection.PNG

Figure 2, shown below, is a snippet from the Analysis Tool Pack’s Summary Level Dashboard.  It shows a plan’s performance relative to its competitors.  In the below graph, you can see that H1112: Competitor 1 has been at 5 stars for its overall score for three consecutive years.  H1114: Competitor 3 has been at 4.5 stars, as has the primary plan, labeled “H0000: Your Plan” below.  H1117: Competitor 6 was at 2.5 stars for Stars 2015, moved to 3.5 stars for Stars 2016, and arrived at 4 stars for its Overall star rating for Stars 2017.  This dashboard makes it easy to see the relative performance for a plan against its competitors, along with the performance trend across three years.  Part C and Part D are also included in this dashboard, making it very easy to demonstrate Stars performance to your executive suite in an easily consumed visual summary. 

Competitor Viz.PNG

As soon as that has been shared, there will come questions.  Where are our opportunities for improvement?  Where did we improve?  Where did we beat our competitors?  Where did our competitors out perform us?  The Stars Performance Analysis Tool Pack has tools to answer those questions and more.

The answer to the first question, “Where are our opportunities for improvement?” can be found on the Stars FractionsTM dashboard, seen in Figure 3 below.  In this dashboard, it is possible to see the star performance for the plan across all measures at once.  This dashboard addresses the issue of showing inverted measures, lower is better, alongside traditional measures, higher is better.  At the same time, it shows the plans relative performance within the star range.  Here we can see that H0000: Your Plan is performing at the upper limit of the 5-star range for the first two measures.  By hovering over a data point, the full name of the measure can be seen along with the relative star score.  In the screen print below, C21: Getting Appointments and Care Quickly was selected.  The Star Fraction score of 5.375 demonstrates that the plan was well into the 5-star range for the selected measure.  The answer to “Where are our opportunities for improvement?” can be found on the right-hand side of the chart.  In this example, the last three measures (difficult to read here but easy to read in the report) are: D15: MTM Completion Rate for CMR (2.034), C18: Reducing the Risk of Falling (2.000), and C04: Improving or Maintaining Physical Health (1.818).  The power of this report in identifying performance at a glance is immense.

Star Fractions.PNG

The next question, “Where did we improve?” can be answered by reviewing two separate reports.  Each report provides slightly different information and has a slider that makes it easy to review each measure within minutes, Figure 4 (to the right). 

CMS Measure.PNG

The first report that answers “Where did we improve?” has a similar layout to the Stars FractionTM dashboard on the previous page.  This is called the Stars FractionTM Measure-Level Report.  Shown in Figure 5 (below), shows the result for C09: Annual Flu Vaccine.  For this measure, the plan demonstrated near linear improvement over three years relative to the cut points.  For Stars 2017, H0000: Your Plan barely achieved 5-stars, indicating that additional improvement needs to be made.  But how did that 5-star cut point change over the three years?  The second report adds that level of detail, but loses the higher-is-always better appeal of the Stars FractionTM Measure-Level report.  The report also shows how H0000 is doing relative to its selected competitors.  Between Stars 2016 and Stars 2017, H000 closed the gap significantly against H1114, which saw only modest improvement relative to the 5-star cut point.

Compare - Figure 5.PNG

The second report that helps answer “Where did we improve?” is the Measure-Level report, Figure 6 (below).  For C09: Annual Flu Vaccine, the 5-star cut point dropped slightly.  That in conjunction with H0000’s increased performance helped achieve the 5-star result for this measure.

Compare - Figure 6.PNG

 The same reports would demonstrate where a plan outperforms its competitors and where its competitors outperformed it.  Below is an example of an inverted measure.  Figure 7 and Figure 8 demonstrate where H0000 outperformed its competitors on the C26: Complaints about the Health Plan (CTMs).  This measure demonstrates how the two reports represent inverted, or lower is better, measures.  Being able to exhibit Figure 7 for all measures makes it much simpler to share this information with individuals who may not have detailed understanding around how the star measures work, avoiding frustration and confusion all around.  Again, Figure 8 has the added value of showing movement for the cut points.  As you can see in Figure 8, this measure became much more challenging for Stars 2016, as noted by the narrower range of cut points and loosened for Stars 2017 again.

Stars Fraction - Figure 7.PNG
Compare - Figure 8.PNG

In addition to these reports that are available within McCormick BI, LLC’s Stars Analysis Tool Pack.  Additional packages are part of our offering.  Your data can be incorporated into their model and compared to projected stars cut points.  Cut points are projected based on trend data, but also industry performance for key metrics where additional information becomes available throughout the measurement year.  This package requires the Stars Analysis Tool Pack to be purchased with additional labor needed to incorporate your internal data into the business model.  With that detail, predictive analytics will provide a mid-year projection of where your organization will be for the upcoming Stars year.  As more data points become solidified and uncertainty is reduced, the predictive result band will decrease.  Executives are thirsty for this level of information and McCormick BI, LLC has the skills and tools to deliver.

Consultative services are also offered to help with driving stars results.  Knowing what to change within the organization or even where to look can be daunting.  McCormick BI, LLC has staff that can pull back the lid and see where in your organization the challenges are located.  Consultative services do not require the purchase of the Stars Analysis Tool Pack, but if purchased it will provide all of the reporting indicated above and more, eliminating the need to create the reporting and allowing you to have this level of detail much more quickly than it could be developed internally.

McCormick BI, LLC has been in business since for over nine years.  Founded by Patrick McCormick, the organization became a Tableau® Partner at its inception.  The company leadership has been in the Stars consulting arena for nearly five years and the data management arena for 25+ years.

Tableau® is the platform upon which McCormick BI, LLC built its Stars Analysis Tool Pack due to its ability to easily represent performance data in an easy-to consume format while allowing analysts to rapidly create variations of existing reports and to slice and dice data in search of quick and valuable insights.  McCormick BI, LLC staff will be available to help get analysts up and running with Tableau® as well as helping them understand the underlying data contained within the dashboards and reports. The Stars Analysis Tool Pack allows health plan leadership to be able to answer questions from senior leadership very quickly after Stars results come out.  Not having this tool pack in a plan’s Stars tool box would be short-sighted.


1          Centers for Medicare and Medicaid Services.  Medicare 2016 Part C & D Star Rating Technical Notes.  Available at:  https://www.cms.gov/Medicare/Prescription-Drug-Coverage/PrescriptionDrugCovGenIn/Downloads/2016-Technical-Notes-Preview-2-v2015_09_02.pdf.  Accessed May 19, 2017.

2      Centers for Medicare and Medicaid Services.  https://www.cms.gov/Medicare/Prescription-Drug-Coverage /PrescriptionDrugCovGenIn/PerformanceData.html, 2017 Part C and D Medicare Star Ratings Data (v07 10 2017 ) [ZIP, 8MB], \2017 Star Ratings Technical Notes, Fact Sheet and Medication Lists\2017_Star_Ratings_Fact_Sheet.pdf. Accessed September 22, 2017.

3      Centers for Medicare and Medicaid Services.  https://www.cms.gov/Medicare/Prescription-Drug-Coverage/PrescriptionDrugCovGenIn /Downloads/Low-Performing-Termination-Letter-04122016.pdf.  Accessed September 22, 2017.

Stars Dashboard.png


Use the form below to request a link to an 8 minute video demo of our Stars Performance Tool Pack. This is a subscription based product that gives you detailed insight into how your organization is doing and where you will get the best bang for your buck. It also provides insights into how you measure up to your competitors. CMS Star ratings are being used in advertising for a competitive edge. On the other end of the continuum a Medicare health plan has been shut down due to low CMS Star Ratings. 

This quick video will give you a glimpse of the product. 

Name *