pricinglessons

Why Conjoint Analysis Results Don’t Always Match Reality

by Michael Mora, 26. August 2013

http://www.relevantinsights.com/why-conjoint-analysis-results-dont-always-match-reality

Over the past year I have been involved in several conjoint studies either for price or feature optimization or both in which clients are faced with many unknowns and struggle to reconcile the research results with the real world. The thing is that conjoint analysis can produce results in which preference shares align closely with market shares if the right information is available and embedded into the experimental design and market simulations. Here are some of the variables that have an impact on how well conjoint results could match reality: Attributes and Levels: This seems to be a basic consideration, but it is often the culprit of misalignment of the conjoint results. Sometimes attributes are simplified when too many levels are possible or simply omitted when they are difficult to represent in a conjoint survey setting, or there are too many attributes for a respondent to consider. Other times, preliminary research, which can pinpoint relevant attributes and levels, is skipped for budgetary or other reasons, and the products are evaluated on attributes and levels that don’t do a good job at representing how products are evaluated in the real world. External Factors: A conjoint analysis may well capture preferences accurately based on the right attributes and levels; however factors such as level of awareness, marketing strategy and tactics, advertising of product benefits, sales process, distribution, category inertia, and competitors’ reaction, among others, are often unknown at the time of the study design or analysis. Any discrepancy between the assumptions made in the research and what’s happening in the marketplace regarding these factors can make conjoint analysis results inaccurate. Sampling Plan: Including the right type of respondents in the study is an important part of the study design, and by that I mean including people who are at least active in the category. Disengaged respondents due to lack of knowledge or experience in the category are not able to provide relevant answers and you may ended up underestimating preference share. Survey Design: The challenge is always to design surveys that mimic how consumers think about the product that is being evaluated to obtain realistic answers. However, this can be difficult when we have to balance the amount and type of information provided, respondent burden, and survey length (which has cost implications) particularly when we are evaluating complex products or services with many attributes and levels, competing alternatives or new products. There are occasions when the list of attributes is reduced to minimize respondent burden and shorten the survey. This could leave relevant attributes out. There are other situations in which to make it easy for the respondent to go through a conjoint exercise that requires a lot of information processing, of either complex or new information or both, respondents are often “primed” by exposing them to the attributes and levels through introductory questions before they get to the conjoint section. In this way when they get there, they are more familiar with the product attributes and can make more thoughtful decisions. However, in the real world, consumers often don’t have the time or will to get acquainted with all the characteristics of a product and make decisions on limited information. “Priming” can also lead to higher awareness of certain attributes than the advertising and marketing strategy and tactics would never produce, which then could lead to inflated importance of those attributes and over-prediction of a product’s preference share. I believe that techniques like conjoint analysis have many useful applications, but should be used with caution and full knowledge of its limitations based on the information available at the time of design and analysis in order to manage expectations on the validity of the results in the real world. I hope these issues don’t discourage you from using conjoint analysis, but make you aware of the information needed and the work required to make its results valuable and actionable.

Comparing Willingness To Pay Measurements To Real Purchase

by Michaela Mora, 14. February 2011

 

I can’t emphasize enough the need for realism in price research in order to get valid results. We are well aware that in the context of a survey, respondents are likely to over- or underestimate their willingness to pay for products and services (more pronounced in certain categories than in others), so we need to strive for research designs that minimize the gap between the survey context and actual purchase behavior.

An interesting study comparing different approaches to price research, with realism in mind, was recently published in the 2011 February issue of the Journal of Marketing Research (How Should Consumers’ Willingness To Pay Be Measured? An Empirical Comparison of State-of-the-Art Approaches by Miller, Hofstetter, Khromer and Zhang).

In the article the authors present a comparative study of direct and indirect methods to measure willingness to pay to actual purchase behavior of a cleaning product. The methods examined include:

1. OE: Direct questions asking consumers to state their willingness to pay using an open-ended question format (Direct)
2. CBC: Choice-Based Conjoint Analysis, which calculates willingness to pay based on consumers’ choices among several product alternatives and a “none” choice option (Indirect)
3. BDM: The Becker, DeGroot and Marschak approach, in which the participants are obligated to purchase a product if the price drawn from a lottery is less than or equal to his or her stated willingness to pay in response to a direct question (Direct)
4. ICBC: Incentive-Aligned Choice-Based Conjoint analysis, in which participants are also obligated to purchase a product if the price drawn from a lottery is less than or equal to the  inferred willingness to pay based on his or her choices among several product alternatives and a “none” choice option (Indirect)
5. REAL: in which participants were asked whether they would buy a product using a dichotomous format (Yes/No) at a certain price displayed in an online shop similar to that of the product manufacturer. This data was treated as real purchase behavior. The prices shown in the online shop corresponded to the price levels used in the conjoint analysis groups (CBC and ICBC).

All participants who were obligated to buy in the BDM and ICBC groups, or agreed to buy in the REAL group, were sent the product and an invoice by mail, which they could pay in cash or by credit card.

The most significant finding of this exercise is that in terms of the absolute distances between an elicited willingness to pay and the actual purchase behavior, the BDM group showed the least deviation from the REAL group, followed by ICBC, OE and CBC. In other words, the BDM and the ICBC approaches elicited a more realistic response to different price points and yield results closer to those of the REAL group, which is not surprising as these approaches put participants in an actual purchase situation.

 However, I was surprised to learn that OE did better than CBC. These results stay in contrast with other studies (Ding, Grewal, Liechty, 2005) and the results from a second dataset the same authors analyzed (digital compact camera) where CBC performed better than OE. The authors hypothesize that CBC is likely to perform better when a product is less unique and faces more competing products, which was not the case of the cleaning product tested. This makes sense and tells us that we should consider the product category when deciding which willingness to pay measurement approach we should use.

 As an overall conclusion, the authors suggest that market researchers should consider incentive-aligned approaches such as BDM or ICBC when:

• A product prototype exists
• The underlying product is not expensive
• The cost of fulfilling the actual buying obligation is low

 Although, the study itself has several limitations (assumptions of market monopoly, small number of CBC tasks, limited to one product category, low completion rates and relatively low understanding of the task under incentive-aligned conditions), it gives support to the idea that we are likely to get more realistic results in price research if we provide test conditions that are as close as possible to the actual purchase situation.

 

http://www.relevantinsights.com/willingness-to-pay-measurement

How to save money with energy-efficient windows? This is a good example of a sales pitch

Interesting sales guide, very fact-driven and credible

The U.S. Department of Energy (DOE) is pleased to provide partners with new cost and energy savings estimates for use in marketing ENERGY STAR qualified windows. Below you will find annual national, regional, and local savings estimates for choosing ENERGY STAR when replacing single pane or double pane, clear glass windows in existing construction or as an alternative to double pane, clear glass windows in new construction, as well as sample factoids based on these estimates.

Download ES_Windows_Cost_Energy_Savings

Summary of the 20 Lessons on Pricing (pdf download)

Open this file in full resolution

#owp2013 pricinglesson #14: There are only two types of discounts. Stupid ones and smart ones. Getting rid of the stupid ones can increase your margin by 1% on average.

Use the price waterfall method by McKinsey to identify and eliminate "stupid" discounts

pricinglesson #8: Research your customers’ willingness-to-pay pricinglesson #WTP: and watch out for the cliffs in the demand curve.

There are two dominant research methods in pricing: conjoint analysis and van Westendorp.

The Price Sensitivity Meter (PSM) is a market technique for determining consumer price preferences. It was introduced in 1976 by Dutch economist Peter van Westendorp. The technique has been used by a wide variety of researchers in the market research industry. The PSM approach was a staple technique for addressing pricing issues for the past 20 years. It historically has been promoted by many professional market research associations in their training and professional development programs. The PSM approach continues to be used widely throughout the market research industry and descriptions can be easily found in many market research websites.

 

#pricinglesson 7: Cash is a fact, costs are opinions. Consider «avoidable» costs instead of variable costs and study margin-equivalent sensitivity scenarios. #owp2013