ARF: First Opinion
An ARF Research Review of Brand Keys'
Brand Engagement Measurement Methodology
William A. Cook, Ph.D.
Sr. V.P., Research & Standards
January 30, 2007
Click Here to view this article as a printable PDF format (728Kb)
Introduction
This is an ARF Methodological Review of the Brand Keys' Brand Engagement measurement methodology. The purpose of the Brand Keys methodology is to provide a measure of the level of brand engagement that results from advertising and communications efforts based on the equity the brand has developed and the brand's combination of channels and messages. Engagement is defined by Brand Keys as "the consequence of any marketing or communications effort (through any media touch point) which results in an increased level of 'brand equity' for the brand". They define brand equity as "the degree to which a brand is believed by the target audience to be able to meet or exceed consumer expectations they hold for the category in which the brand competes".
The purpose of this review is twofold:
To review the objectives, design, methodology and reporting of the Brand Keys methodology in the context of ARF guidelines.
To render an opinion of the adequacy of the design, methodology and reporting to meet the stated objectives, both in theory and in practice, insofar as that practice is represented to the ARF by Brand Keys.
The ARF opinion is based on the years of experience of the ARF staff and the body of ARF principles and guidelines currently available.
Objective
The objective of the Brand Keys methodology is to provide a measure of the level of engagement that a) is possessed by the brand and/or b) results from advertising and communications efforts based on the brand's combination of channels and messages. The basis for the Brand Keys assessment is a set of predictive loyalty metrics which they have developed specifically for this purpose.Design
The methodology is designed to assess the brand equity for a given brand based on a survey of the brand's target audience by measuring the brands in the subject category, and the ideal brand on a variety of category attributes, benefits and value components. The target consumers also indicate the importance of those attributes, benefits and values to them.
It seeks to provide the marketer with actionable answers to five basic questions:
- What are the four primary loyalty/engagement drivers for my category, i.e., how do respondents view the category, compare offering in the category, and, ultimately, behave positively toward the brand being assessed?
- Which specific key attributes, benefits, and values form the components of each driver?
- What's the order of importance of the drivers?
- What expectations do consumers hold for each driver?
- How does my brand stack up to a category Ideal?
Methodology
Project Initiation. A Brand Keys executive consults with the client to develop:
- A definition of the target consumer.
- A list of category attributes, benefits and values thought to be relevant to the target customer, and therefore to the brand marketing process.
- An assessment of the business issues that prompted the request for the research project and a consensus on the action steps for marketing depending on alternative outcomes from the current research.
Survey Design. A survey is conducted among 50 to 500 members of the target audience depending on the number of targets, variables (including competitors), under study. The survey includes a series of questions to establish the importance of selected attributes, benefits and values to the target consumers. It also includes a proprietary battery of psychological questions derived from Jung's theory of personality types as extended by Briggs and Myers. Because the psychological question set has been demonstrated to have high test/re-test reliability (.93) in two national probability sample studies – one in the US and another in the UK, Brand Keys is able to provide valid metrics utilizing smaller sample groups. The psychological questions establish consumers' expectations for products in the selected category including an Ideal brand, the clients' brand and key competitive brands.
Sample Design/Selection. Sample design and selection is based upon accepted practices within the marketing/research communities. Discussions are held with clients to determine the optimum target segment(s) that is both most appropriate and will afford insights necessary to provide "answers" to clients' questions.
Traditional screening techniques – including security questions – are utilized. Quotas (as necessary) are monitored by a dedicated project Field Director and/or CRT programming. Validation of 15% of the interviews is standard operating procedure.
Brand Engagement Analysis. Four drivers, or factors, are derived from the consumer expectations measures obtained from the psychological questions.1 Each driver identifies its importance to the average target consumer, and the expectations that the target consumers hold for the category, as well as, similar measures of individual brands, and the ideal brand. The expectations of a brand are expressed as an index relative to the ideal brand, with brand loyalty and engagement measured vis à vis the degree to which the brand/test variable is able to meet or exceed customer expectations held for the category Ideal.
The driver factors are statistically related to the collection of attributes, benefits and values to help define the drivers from a consumer perspective.
A causal path analysis is used to tease apart the driver weights and the individual attribute, benefit, or value components. The path coefficients reveal how much each component contributes to the overall level of brand engagement or loyalty.
Reporting. Brand assessments are presented to the client via charts displaying the four drivers in the order of their importance to their customers across the horizontal axis and the level of consumer expectation for each driver is identified on the vertical axis. (Figure 1)
The selected brand can be shown in comparison with the Ideal brand and/or the selected brand when shown to consumers in a given media (or marketing) context.
Figure 1. Ideal Pizza Drivers
Figure 2. Comparison of Ideal Pizza with Domino's
As with the emphasis on client involvement at project initiation, the reporting and client discussion of findings is carefully tailored to the current business needs of the client, and the analysis is detailed accordingly.
Validation Evidence
Numerous of validation studies have been provided by Brand Keys at national and international symposia. Several are available on their website: www.brandkeys.com.
Krispy Kreme Donuts Case. One dramatic example resulted from their tracking of Krispy Kreme Donuts during the period in which their sales experienced a dramatic decline (2003-2004). Brand Keys metrics accurately predicted the decline due to unmet consumer needs/expectations six months prior to the company reporting the declining same-store sales figures. The correlation between the Brand Keys brand equity rating and the change in per store sales was .97.
Brand Keys Validation of Brand Engagement. Validation evidence has been accumulated from a series of studies in the US, Mexico and Argentina. The Brand Engagement scores for television channels/programs were shown to be significantly positively correlated with in-market behavior:
- Retailer store visits, spending per visit, and total spending in the US r = .87, .85, .99
- Purchase of a bread/bakery products brand in Mexico (# items of brand, average price/item, average total spent on brand) r = .87, .84, .94
- Purchase of a beer brand in Argentina r = .88
Brand Keys Validation Brand Keys demonstrated the sensitivity and stability of their method through the use of a "Stacked Bars Comparison" to show the efficacy of utilizing smaller sample segments than are required with traditional, direct-inquiry question formats. In the foundation survey conducted in the US and UK, 1,600 respondent assessments in each market were "compared" utilizing groups of 10 respondents, i.e., 10 respondent assessments were analyzed.
Then an additional 10 respondents were added to the first 10 respondents and that data was analyzed – and compared to the Group #1 assessments. Additional groups of 10 respondents each were assessed and added to the previously analyzed groups. The analysis showed that there were no significant differences identified until total sample segments had been increased to 380 respondents.
In addition to the ongoing validation assessment by Brand Keys, two other parties have shared validation analyses that they have conducted. These are summarized here, followed by several examples produced by Brand Keys.
Aquetong Capital Advisors LLC shared with Brand Keys a comparison of Brand Keys' Brand Valuation Ranking and their composite of two financial performance ratios used by Aquetong analysts: Total Enterprise Value (TEV)/EBITDA and TEV/Revenue. (TEV is the sum of market capitalization and outstanding debt. EBITDA is an approximate measure of a company's operating cash flow – Earnings Before Interest, Taxes, Depreciation and Amortization.) Aquetong analysts found very strong correlations between their composite TEV score and Brand Key's ranking for brands in each of 10 categories. The squared correlation coefficients displayed in Table 1 reveal the percent of variation in the Aquetong measures for each brand accounted for by the Brand Keys measures.
| Category | R2 |
| Energy | .83 |
| Office Equipment | .84 |
| Technology | .84 |
| Restaurants | .85 |
| Telcom | .86 |
| Wireless Providers | .87 |
| Retailers | .87 |
| Parcel Delivery | .89 |
| Airlines | .90 |
Northwestern University's Forum for People Performance Management and Measurement. Professor Frank Mulhern of Northwestern University and Don Schultz and Heidi Schultz of Agora, Inc. conducted a study linking employee and customer behavior to financial outcomes in cooperation with Brand Keys. They studied the guest visitation and spending behavior for a large international hospitality brand's premier tier customers. One of the four psychological drivers, Tries to Satisfy, was found to correlate highly with Total Spending Dollars and Spending per Visit. This driver had been identified by Brand Keys as the driver in which consumers held the highest expectations.
ARF Opinion
Overview
The disciplined application of the Brand Keys approach to measuring brand equity, customer loyalty, together with their engagement theory of brand equity and their survey, produce both a consistent set of findings and overarching principles to guide their clients' marketing activities. This is especially important because of the complexity of the statistical tools employed to produce the metrics and reports embodied by the Brand Keys approach.
The Brand Keys method provides clear and reliable metrics for assessing brand engagement and brand loyalty. By focusing on consumer expectations, Brand Keys provides detailed guidance for brand marketers seeking to increase brand loyalty and profitability.
Relevant ARF Principles and Guidelines
The Guidelines for Market Research published by the ARF in 2002 provide a valuable guide for researchers seeking to produce high quality surveys, behavioral measurements, data analyses and reports. A comparison of the Brand Keys methodology to relevant guidelines from that document is provided in the table below.
| Brand Keys Compliance | |
| Clients should be involved in setting the objectives for the research and the objectives should be clearly linked to the business needs. In addition, the returns on the research investment should be documented for the clients. | Brand Keys employs a program of educating first-time clients on the relationship between measuring and monitoring brand engagement to building brands and to maintaining profitable customer loyalty. Client input is thoroughly sought on:
|
| Survey design should minimize bias and consumer burden while maximizing the quality of the information and its relevance to the study's objectives. | The psychological assessment questions in the The psychological assessment questions in the measurement of brand engagement have been carefully developed, assessed for test/re-test reliability and externally validated in multiple categories and countries.
Extraneous questions are avoided and the duration of the interviews are kept to 10-20 minutes in the majority of surveys. |
| In sampling, the population definition should correspond closely to the key prospects, customers, or target audiences who are the focus of the business issue or inquiry. | Care is taken in working with the client to specify the appropriate population from which to draw the survey sample. In addition, the sample frame development is executed with diligence. In addition, the selection of the survey mode and the execution of the survey are conducted in a way to maximize the likelihood that the respondent is in that population. |
| In reporting the research conclusions must be represented clearly, objectively, and accurately.accurately. | Reporting is simplified by the use of simple bar charts built around the four drivers presented in the order of their importance to the consumer. The brand attributes and benefits and associated values are also presented in the context of the four drivers. |
| It is important to retain the integrity of the research findings by separating the findings from the interpretation and recommendations. from the interpretation and recommendations. | Interpretation of findings and recommendations are delivered as the experienced judgment of Brand Keys executives and are thoroughly Brand Keys executives and are thoroughly |
Brand Keys Policies and Stakeholders
Brand Keys, Inc. is a privately held research company with offices in New York, NY. The Robert Passikoff, Ph.D. is the President of Brand Keys. Dr. Passikoff is the recent author of Predicting Marketing Success: New ways to predict customer loyalty and engage customers with your brand. Brand Keys was founded by him in 1984.
According to Dr. Passikoff, "Brand Keys specializes in predictive loyalty and engagement metrics that accurately forecast future consumer behavior. We believe that insights and assessments should correlate with sales and profitability."
Current Clients
ABC TV
Ann Taylor
Body Shop
Cablevision
Calvin Klein
Discover Financial
KeySpan Energy
Knight-Ridder
Masterchem
Neutrogena
OfficeMax
Press-Enterprise
Samsung
Scottrade
The New York Times
Toyota
Unilever Best Foods
Universal McCann
XM Satellite
Advisory Board
Marc E. Babej, President, Reason Inc.
Joseph Dell'Aquilla, Managing Partner, Continental Consulting
Jack Kaplan, Associate Professor, Entrepreneurship, Columbia University
Kerry O'Connor, General Manager, Mirror Group Newspapers, London, England
Don E. Schultz, Ph.D., Professor Emeritus-in-Service of Integrated Marketing Communications,
Medill School of Journalism, Northwestern University and President, Agora Consulting
Barry Sheehy, President, CPC Econometrics
Leonard Stein, President, Visibility Public Relations
Jack Trout, Founder, Trout & Partners
| Ethical Principles | Brand Keys Compliance |
| Privacy policies. Researchers must respect the rights of the individual to anonymity and privacy. Researchers must also reasonably ensure that any confidential information provided to them is protected against unauthorized access. |
Brand Keys is diligent in the selection of the field services that it selects, and maintains strong relationships and oversight with the management of those firms. Among the research quality assessments that are consistently monitored is respondent anonymity and privacy protection. |
| Fully disclose methodology. Complete information about research methods and practices used, as well as all the data collected, and its ownership, should be revealed to all research subscribers and prospective subscribers. …all methods used should be as "transparent" as possible, thereby permitting critical evaluation and replication. The details of disclosure should include the following at a minimum:
|
In-depth client briefings prior to initiation of the study include a review of: the survey design and questionnaire; sample design and procedures; analytic methods and procedures; and reporting process and materials. We recommend that the Brand Keys compile its technical documentation into a formal Technical Guide covering the details of disclosure noted in the left column here. |
| Research companies must take steps to ensure the responsible use of their data in the public domain – among clients, the press, and others likely to cite their results in public contexts. | Brand Keys, Inc. provides both the press and companies seeking to utilize results of such studies within a public context with supporting and/or audit data to ensure responsible application of findings. In addition, Brand Keys adheres to the codes of ethics set by the leading research organizations throughout the world in order to ensure responsible use of data and findings within all contexts. Brand Keys is scrupulous in respecting and guaranteeing respondent confidentiality. |
References
Robert Passikoff, Predicting Marketing Success: New Ways to Predict Customer Loyalty and Engage Customers with Your Brand, 2006, John Wiley & Sons.
Bob Lamons, "Sometimes Indirect Route Is Shortest Path," Marketing News, February 15, 2006, p. 6-7.
Robert Passikoff, "Winning Strategies for Consumers of All Ages," Research World, April 2005, pp. 34-35.
,"Brand-to-Media Engagement," Integrating Engagement Research Into Business, ESOMAR, September 19, 2004.
,"Let's Move Research into the 21st Century," Revolution, March 2000, pp. 15.Acknowledgements
We gratefully acknowledge the contribution of the industry experts who reviewed this document and provided several useful suggestions to the author:
Laura Bright – Consultant
Mark Loughney – VP, Sales & Strategy Research – ABC Television Network
Beth Uyenco – Director, Research MSN – MSN
Charlene Weisler – SVP, Director of Research – Rainbow AdvertisingClick Here to view this article as a printable PDF format (728Kb)
1 Four factors usually account for over 85% of the variance, and adding more factors was judged by Brand Keys developers to add complexity but not explanatory value.