«By JONATHAN D. KETCHAM, NICOLAI V. KUMINOFF AND CHRISTOPHER A. POWERS* We develop a structural model for estimating the welfare effects of poli- cies ...»
In words, an informed consumer will never choose a plan that has higher cost, higher variance, and lower quality than some feasible alternative. We refer to choices that satisfy (1.a)-(1.d) as being dominated. In theory, a consumer may choose a dominated plan if she is risk loving, if she dislikes quality, if she has a negative marginal utility of income, or, more likely, if she is misinformed about her options. Hence, if we observe a consumer actively choosing a dominated plan then we label her choice as “suspect”. We suspect that the consumer is misinformed and, therefore, that her enrollment decision may not reveal 15 Similar to Chetty et al. (2015) we define an enrollment choice as active if either of the following statements is true: (1) the person is new to the market and must select a plan to become insured or (2) the person switched to a new plan during open enrollment. If neither statement is true, then the enrollee took no action during open enrollment and was automatically reenrolled in the plan she chose last year—her default—in which case we define her choice as passive. After the inaugural enrollment cycle in 2006 between 20% and 23% of enrollees made active choices each year (Table A1).
11 her preferences.16 To test whether enrollees chose dominated plans we define cost, variance, and quality using methods from the literature on PDP choice (Abaluck and Gruber 2011, Ketcham, Kuminoff, and Powers 2015). First we assume that informed consumers have unbiased.17 Next, we use a
expectations of their drug needs for the upcoming year:
cohort approach to calculate variance. We calculate from the distribution of expenditures under plan j for the drugs used in year t by people in consumer i's cohort in terms of year t-1 drug claims. Specifically, we use CMS’s random 20% sample of all PDP enrollees to assign each individual in the MCBS sample to 1 of 1000 cells defined by the deciles to which she belonged in the national distributions of the prior year’s total drug spending, days’ supply of branded drugs, and days’ supply of generic drugs.18 Then we calculate for the distribution of drugs used by everyone in consumer i’s cell. Finally, we allow utility to depend on indicators for insurance companies. These indicators reflect all aspects of PDP quality that vary across insurers, such as customer service, pharmacy networks, mail order options, and prior authorization requirements.19 Because we allow utility to depend on insurer dummies, a chosen plan will be dominated if and only if the enrollee could have chosen a different plan offered by the same insurer that would have lowered the mean and variance of her drug expenditures, or lowered one holding the other constant. The first row of Table 3 shows that 19% of beneficiaries actively enrolled in dominated plans in the first year of the program, when everyone had to actively enroll. The share ranged from 4% to 7% in subsequent years. The decline after 2006 is mostly due to a decline in active decision making. That said, the probability of choosing a dominated plan conditional on making an active choice also deConsumers who violate at least one condition are choosing plans on what Lancaster (1966) called the “efficiency frontier” in attribute space. Every plan on the frontier can be rationalized as maximizing some utility function that satisfies the preference axioms and weak risk aversion under full information. For example, an informed risk averse consumer may optimally choose a more expensive and lower quality plan that better insures her against negative health shocks.
17 Our econometric estimates and policy conclusions are robust to assuming that consumers are myopic:. This is unsurprising since individual prescription drug use is strongly persistent over time.
18 In cases where CMS did not have the person’s drug claims from the prior year, such as 2006, we predicted their deciles based on current and future drug claims and past, current and future health.
19 For example, stringent prior authorization requirements for certain drugs may be unattractive to consumers who believe they have a high likelihood of purchasing those drugs and irrelevant to consumers who do not. Likewise, consumers differ in their proximity to innetwork pharmacies. These factors vary across insurance brands and consumers but not across plans within a brand.
12 clined by three percentage points between 2006 and 2010.
To hedge against potential Type II error in using active choices of dominated plans to identify misinformation, we use an MCBS knowledge question to develop a second suspect choice indicator. Each year, respondents were asked to state whether the following sentence is true or false. Your OOP costs are the same in all Medicare prescription drug plans. For people with no drug claims, the statement is true. For people with any claims the statement is false due to variation in formularies, deductibles, and coinsurance. Understanding that drug costs vary across plans is the central to understanding how the market works.20 Moreover, this variation is financially important: the average beneficiary’s OOP costs for her purchased drugs vary by over $1,100 across her available plans.
Suspect choices (union of the first four rows) 55 52 48 40 39 44 Note: The table reports the share of choices triggering each indicator, by year. The MCBS knowledge question asks whether the enrollee’s out of pocket costs are the same under every available drug plan. The correct answer is coded as yes for enrollees who filed drug claims in both the prior and current years if their out of pocket costs did in fact vary across plans in both years. The last row reports the share of enrollees satisfying the criteria in either of the first two rows. See the text for additional details.
We use each person’s drug claims to determine their correct answer to the MCBS question. Because respondents may be unsure about which enrollment year the question is referring to, we code a person’s answer for year t as correct if it is correct for either year t or year t-1. Table 3 shows that 44% of respondents gave the wrong answer in 2006 and between 7% and 9% gave the wrong answer when making active choices in subseThe MCBS asks five other questions that test knowledge of Part D, but they are less relevant for forecasting individual drug expenditures. Howell, Wolff and Herring (2012) provide further analysis of the MCBS knowledge questions.
13 quent years.21 On average, respondents who answered incorrectly could have saved 14% more by switching to a different plan than those who answered correctly.22 Finally, when beneficiaries are passively reenrolled in their default plans we defer to their preceding active choices of those plans when coding their passive reenrollment decisions as suspect or non-suspect. Table 3 shows that from 2007 to 2010 12% of beneficiaries were passively reenrolled in plans that were dominated when they were actively chosen and 27% were reenrolled in plans that were actively chosen during enrollment cycles in which they answered the knowledge question incorrectly.
In summary, we define a choice as suspect if the decision maker (i) actively enrolled in a dominated plan; (ii) actively enrolled in a plan while answering the knowledge question incorrectly; or (iii) passively reenrolled in a plan that satisfied (i) and/or (ii) at the time it was first chosen. The last row of Table 3 shows that 55% of all enrollment decisions satisfied at least one of these criteria in 2006 and 44% between 2007 and 2010. In later sections we demonstrate that our policy conclusions are robust to alternative ways of defining suspect choices. This includes focusing exclusively on dominated plan choices;
using a more inclusive definition that adds enrollees who could have reduced their expenditures by more than 33%; and assuming that consumers are myopic in the sense that they expect their drug use in the upcoming year to be identical to the prior year. These and other robustness checks are described in section VII and the appendix.
To develop intuition for potential mechanisms driving suspect choices, we estimate linear probability models in which the dependent variable,, is an indicator for whether person i in CMS region r made a suspect choice in the year t enrollment cycle,
21 Aggregating over active and passive choices, the total share of respondents answering incorrectly was 29% in 2007, 31% in 2008, and 28% in 2009 and 2010. The 15 percentage point reduction between 2006 and 2007 is consistent with prior evidence on learning in PDP markets (Ketcham, Lucarelli, and Powers 2015, Ketcham et al. 2012).
22 In about 10% of our sample the person who responds to the survey and makes the enrollment decision is a proxy for the beneficiary, such as a spouse or child (Table A1).Table A2 shows that when we focus on beneficiaries who made their own active enrollment decisions, answering the knowledge question incorrectly is associated with a two percentage point increase in the probability of choosing a dominated plan conditional on demographics, year and CMS region (p=.172). It is also associated with a $24 increase in the amount of money the beneficiary could have saved by switching to a different plan offered by the same insurer (p=.011).
Note: The table reports coefficients and standard errors from linear probability models of individual’s plan choices. The dependent variable equals one if we suspect the choice was misinformed. See the text for a formal definition. All explanatory variables are binary expect the number of available plans and the number of drug claims, both of which are standardized. The omitted indicators define the baseline enrollee as a 65 to 69 year old white male who did not finish high school, has income below $25k, does not get help making insurance decisions, has not searched for CMS information using the internet or 1-800-Medicare, has the mean number of drug claims, and has not been diagnosed with dementia or depression. All regressions include indicators for enrollment year and region. Robust standard errors are clustered by enrollee. *,**, and *** indicate the p-value is less than 0.1, 0.05, and 0.01.
The first column of Table 4 reports results for enrollment decisions from 2006-2010.
23 These indicators capture variation in the complexity of choice sets across space and time. For example, in the first year of the program the number of available plans per region ranged from 27 to 52. The number of plans also changed over time, increasing noticeably between 2006 and 2007. This variation allows us to test the choice overload hypothesis that consumers are less likely to make informed decisions as the number of options grows. Ketcham, Lucarelli and Powers (2015) test choice overload in Part D more extensively, and capitalizing on individual-specific variation in the number of plans available by relative cost of those plans.
15 The omitted indicators define the reference person as a 65 to 69 year old unmarried and retired white male with no high school diploma who has not searched for information on CMS programs and makes his own enrollment decisions. The coefficients imply that obtaining a college degree is associated with a 6.2 percentage point reduction in the probability of making a suspect choice. The probability is higher for nonwhites (+13.5) which might proxy for unobserved differences in wealth or education, and it is higher for people who get help making decisions (+2.8). We also see lower probabilities for enrollees who searched for information about CMS programs using the internet (-7.3) or by called 1Medicare (-4.4).24 Looking at the administrative variables, the probability of making a suspect choice is increasing in age, consistent with prior evidence on the decline in cognitive performance for individuals over 65 (Agarwal et al. 2009, Tymula et al. 2013). The predicted probability is approximately 6 percentage points higher for enrollees in their late 70’s and approximately 14 percentage points higher for enrollees in their late 80’s. This is after controlling separately for diagnosed cognitive illnesses normally associated with aging, namely dementia (+4.7) and depression (+3.7), and conditioning on the increased complexity of decisionmaking associated with greater drug needs via a measure of total drug claims (+3.4 for a one standard deviation increase in claims). In comparison we find that income, gender, marital status, and the existence of children have small and statistically insignificant effects. We also obtain a precisely estimated zero on the number of available plans, providing evidence against the hypothesis that choice overload causes suspect choices.
The last column of Table 4 shows that the results are largely unchanged if we drop
2006. We exclude 2006 enrollment decisions from our main analysis because of the sharp improvement in knowledge question responses in 2007. Because consumers appear to have learned during the inaugural year of the program, their choices in that first year may be less informative for analyzing prospective policies. That said, we show in the appendix that our main findings are invariant to whether we include or exclude 2006 choices.
24 The later result is consistent with Kling et al.’s (2012) audit of the Medicare help line in which actors calling the number for information found that customer service representatives consistently identified low-cost plans based on the actors’ fictional drug needs.
16 IV. A Parametric Model of Decision Making with Heterogeneity in Beliefs
To evaluate the welfare effects of prospective polices we must select a parametric approximation to utility. The novelty of our approach to is to allow for heterogeneity in beliefs about plan attributes. We focus on identifying parameters that describe how plan attributes affect plan choice and then use our indicators for suspect and non-suspect choices to guide how we interpret those parameters and use them for welfare analysis.
A. Initial Enrollment Decision
When a beneficiary first enters the market in year 0 she must actively choose a plan to obtain insurance. She will choose the plan that maximizes her utility, conditional on her beliefs about plan attributes. We approximate this process with a model similar to the ones used by Abaluck and Gruber (2011), Kling et al. (2012) and Ketcham, Kuminoff, and Powers (2015),