08122021

Back Eşti aici:Home Dezbateri Dezbateri actuale Măsurare şi evaluare în PR Beware invalid metrics

Beware invalid metrics

Pin It

Jim Macnamara websiteThere are some invalid metrics presented as purported evaluation that need to be avoided. The public relations industry is one of the worst offenders in this regard. Most notably, PR practitioners have long calculated alleged advertising value equivalents (AVEs), also referred to as advertising value equivalency and equivalent advertising value (EAV).

These are calculated by multiplying the space gained as editorial publicity in print media and the time gained in news, current affairs and talk shows in broadcast media by the advertising rate for the respective programs and publications. Some even apply multipliers of the cost of an equivalent amount of advertising space or time, arguing that editorial publicity is more credible than advertising. Use of multipliers in the PR industry ranging from 2 to 8 times have been reported by Walter Lindenmann (2003) and Mark Weiner and Don Bartholomew (2006).

Research studies over several decades have failed to substantiate so-called advertising value equivalency of editorial publicity, and studies have been particularly condemning of claimed multipliers (e.g., Cameron, 1994; Hallahan, 1999). In his guidelines for measuring and evaluating PR, Lindenmann reported that reputable researchers regard “such arbitrary ‘weighting’ schemes aimed at enhancing the alleged value of editorial coverage as unethical, dishonest, and not at all supported by the research literature” (Lindenmann, 2003, p. 10). Research by Samsup Jo (2004) found that when strong factual and logical arguments are available, editorial publicity outperforms advertising, but when weak arguments are available and persuasion based on emotion or other factors is required, advertising outperforms editorial.

Recently, PR researchers David Michaelson and Don Stacks have conducted several experiments to test the impact of similar advertising and editorial publicity. Following a pilot study (Michaelson & Stacks, 2006), Michaelson and Stacks conducted an experiment in 2007 that involved production of quality print advertisements and mock-ups of editorial coverage in a range of newspapers including The New York Times for a fictitious product, Zip Chips. Professional designers and writers were used to simulate actual advertisements, and editorial coverage and layouts mirrored the style and format of food sections of the relevant media. Groups of potential consumers were then exposed to the advertising and editorial content promoting Zip Chips. This experiment found no significant differences in awareness, purchase intent, or believability resulting from equivalent amounts of advertising and editorial publicity exposure (Michaelson & Stacks, 2007).

Responding to some incredulity and criticisms of their initial studies, Stacks and Michaelson repeated and extended the experiment in 2009 from which they concluded:

There are differences between advertising and editorial commentary but, as in the earlier study, these differences are not the difference expected. What we found was that both the editorial and the advertisement were equally effective in promoting the product, but no statistically significant differences existed between the editorial and the advertisement across measures of awareness, information, intent to purchase, and product credibility. (Stacks & Michaelson, 2009, p. 12)

One difference found was that editorial publicity contributes to higher levels of product knowledge – not surprising because editorial is usually a longer format than ads and thus contains more information. But Stacks and Michaelson concluded: “We still failed to find a ‘multiplier effect’” (2009, p. 15).  And, even though these studies appear to show equivalency of advertising and editorial publicity, it must be borne in mind that they were testing carefully crafted editorial that was 100% positive in relation to the product promoted. In reality, at least four factors make generalized AVE calculations invalid and spurious as a measure of the value of PR/publicity as follows.

  • The figure calculated is not the value of advertising – it is the estimated cost if the same amount of space and/or time was bought as advertising.
  • Most organizations would not purchase advertising in many of the media that publish or broadcast editorial content. Advertising is usually focussed on priority media, while editorial content can be syndicated widely, including in less relevant media (Jeffries-Fox, 2003; Macnamara, 2000).
  • In most instances, editorial and advertising are not equivalent. Whereas advertising is controlled in terms of content and placement, editorial media coverage is highly variable in terms of placement and presentation. Editorial media content may not include key messages and can even be critical and negative in relation to the organization or issue concerned. Editorial articles also often include information about competitors, sometimes even favourably comparing competitors or alternatives.
  • The rates used in such calculations are often casual advertising rates, which inflate so-called ‘values’ because advertising campaigns usually access special rates – often at significant discounts in today’s age of fragmenting audiences and cash-starved media organizations.

Even more recent experimental research by Julie O’Neil and Marianne Eisenmann (2016) has tested similar content in the full range of paid, earned, shared, and owned media (the PESO model), including native advertising, and again found that earned editorial publicity, while important, does not have greater impact on awareness, knowledge, interest, intent to purchase, or advocacy. O’Neil and Eisenmann’s experiment using various types of promotional content for a smartphone exposed to 1,500 participants found that the lines between various media content formats are blurring and that consumers access a range of sources of information. Overall, they found that media consumers today put less emphasis on content format and more on message quality wherever it appears, with reviews written by other consumers being the number one preferred source of information (O’Neil & Eisenmann, 2016).

The Barcelona Principles developed by the International Association for Measurement and Evaluation of Communication (AMEC, 2015) and endorsed by professional communication bodies worldwide, states emphatically that AVEs are not a valid measure of public relations.

At conferences and summits on evaluation, PR practitioners continue to argue that they have to provide AVEs because their clients or bosses ask for them. However, other professionals such as lawyers, auditors, and accountants are sometimes asked to do things that are improper and unethical, such as avoiding tax. Ethical professionals decline to provide such services and advise their clients of alternative approaches based on their expert knowledge. Public communication practitioners using AVEs need to consider whether they want to be recognized as professionals offering expert services, or simply providers of low-end commodity products of dubious quality.

***

Extract from: Macnamara, J. (2018). Evaluating Public Communication: Exploring New Models, Standards, and Best Practice. Abingdon, UK: Routledge (pp. 148–150).

***

Jim Macnamara is a Distinguished Professor in the School of Communication within the Faculty of Arts & Social Sciences at the University of Technlogy Sydney (UTS). He is currently serving as Deputy Dean of the UTS Faculty of Science.

He is also a Visiting Professor at The London School of Economics and Political Science (LSE), Media and Communications Department, and a Visiting Professor at the London College of Communication (LCC) in the University of the Arts London (UAL).

Jim is internationally recognised for his research into evaluation of public communication including advertising, public relations, and marketing, corporate, and government communication to identify effectiveness and inform strategy, and his research into organisational listening has been described as "of major international significance". He also has conducted extensive research into health communication, including for the Cancer Institute NSW, the NSW Department of Health, the NSW Multicultural Health Communication Service, and he has led global evaluation research for the World Health Organization (WHO) during the COVID-19 pandemic and for World Health Days in 2020 and 2021.

His research has had substantial industry, professional, and social impact, including adoption of his evaluation framework for communication by the NSW Government, the International Association for Measurement and Evaluation of Communication (AMEC), and the World Health Organization (WHO), and his recommendations on evaluation and organisational listening have been adopted by the UK Government Communication Service (GCS) in the Cabinet Office, Whitehall, and the European Commission Directorate-General for Communication (DG COMM).

In 2017 he was presented with The Pathfinder Award,"the highest academic honor" awarded by the Institute for Public Relations (IPR) in the USA for his scholarly research, and The Don Bartholomew Award for "outstanding service to the communications industry" by the London-based International Association for Measurement and Evaluation of Communication (AMEC).

Jim is the author of more than 80 academic journal articles and book chapters, a number of influential research reports, and 16 books including, most recently, 'Beyond Post-Communication: Challenging Disinformation, Deception, and Manipulation' (Peter Lang, New York, 2020); 'Evaluating Public Communication: New Models, Standards, and Best Practice' (Routledge, UK, 2018); 'Organizational Listening: The Missing Link in Public Communication' (Peter Lang, New York, 2016); and 'The 21st Century Media (R)evolution: Emergent Communication Practices' (Peter Lang, New York, 2014).

References

Cameron, G. (1994). Does publicity outperform advertising? An experimental test of the third-party endorsement. Journal of Public Relations Research, 6(3), 185–207.

Hallahan, K. (1999). No, Virginia, it’s not true what they say about publicity’s ‘implied third-party endorsement’ effect. Public Relations Review, 25(3), 331–350.

Hoofnagle, C., & Meleshinsky, E. (2015). Native advertising and endorsement: Schema, source-based misleadingness, and omission of material facts Technology Science. http://techscience.org/a/2015121503

Jeffries Jeffries-Fox, B. (2003). Advertising value equivalency (AVE). Gainesville, FL: Institute for Public Relations. Retrieved from http://www.instituteforpr.org/wp-content/uploads/2003_AVE1.pdf

Jo, S. (2004). Effect of content type on impact: Editorial vs. advertising. Public Relations Review, 30(4), 503–512.

Lindenmann, W. (2003). Guidelines for measuring the effectiveness of PR programs and activities. Gainesville, FL: Institute for Public Relations. http://www.instituteforpr.org/wp-content/uploads/2002_MeasuringPrograms.pdf  

Macnamara, J. (2000). The ad value of PR. Asia Pacific Public Relations Journal, 2(1), 90–103.

Michaelson, D., & Stacks, D. (2006). Exploring the comparative communications effectiveness of advertising and media placement. Gainesville, FL: Institute for Public Relations. http://www.instituteforpr.org/advertising-media-placement-effectiveness 

Michaelson, D., & Stacks, D. (2007). Exploring the comparative communications effectiveness of advertising and public relations: An experimental study of initial branding advantage. Gainesville, FL: Institute for Public Relations. http://www.instituteforpr.org/wp-content/uploads/Michaelson_Stacks.pdf  

O’Neil, J., & Eisenmann, M. (2016, March). How changing media formats impact credibility and drive consumer action. In B. Yook, Y. Ji, & Z. Chen (Eds.), Refereed Proceedings of the 19th International Public Relations Research Conference, Miami, FL. http://www.instituteforpr.org/wp-content/uploads/IPR19-proceedings.pdf

Stacks, D., & Michaelson, D. (2009). Exploring the comparative communications effectiveness of advertising and public relations: A replication and extension of prior experiments. Public Relations Journal, 3(3), 1–22.

Weiner, M., & Bartholomew, D. (2006). Dispelling the myth of PR multipliers and other inflationary audience measures. Gainesville, FL: Institute for Public Relations.   http://www.instituteforpr.org/wp-content/uploads/Dispelling_Myth_of_PR_Multiplier.pdf

Pin It

Parteneri

   BT Logo Aliniat Central    GFPR new logo web            logo SMP          chapter4 logoSigla Action Global Communications web dccom LOGO nou siteGMP logo 2        Logo ThePublicAdvisors  logo oxygen site