Katie Paine: The biggest challenge we face is to measure what matters, not just what is easy

katie_delahayeKatie Paine is a renowned expert on public relations and social media measurement. She is the CEO and founder of KDPaine & Partners LLC and author of "Measuring Public Relationships: The data-driven communicator's guide to measuring success." She also writes the first blog (Katie Paine's Measurement Blog) and publishes the first newsletter (The Measurement Standard) dedicated entirely to measurement and accountability. PR Romania had the opportunity to conduct an interview with the Queen of Measurement.

Katie, where did you start, and how did you become involved in strategic communications measurement and evaluation?


I started measuring results back in 1982 when I was the Director of Marketing Communications at Fujitsu and realized that I needed numbers charts and graphs if I was going to my budget approved. I did a detailed media analysis plus a competitive analysis and voila, budge was approved. Then I went to Hewlett Packard and demonstrated that a $15,000 direct mail piece we did resulted in $1.4 million in new sales, then went to Lotus Development and in 1986 essentially invented what we know of today as PR measurement. I hired someone who was in the market for software and told him to read 2000 articles and for each one tell me whether it left him more or less likely to buy the product and if it contained one of our key messages. I then further analyzed the results to determine our best, most effective spokespersons and tactics had been. In 1987 I started The Delahaye Group and the rest is history.

How would you describe the history of public relations measurement and evaluation?


I kind of just described it above. Essentially a few companies like AT&T were doing some rudimentary PR measurement prior what I was doing at Lotus, but there was really nothing else when I started.  Eventually there were a handful of us doing PR measurement and that part of the business hasn’t changed much.

In 1990 we added tradeshow, event and sponsorship measurement to the mix, and that became part of PR measurement. In 1992 with the formation of the EU you saw an explosion in international measurement, and we began to create DelNet, which was the first true international network of measurement firms. In 1996 we began analyzing what at the time would be called “The Internet” but what we know of today as consumer generated media and web sites.  In about 1999 the first automated analyses came along with Biz 360. That was still mostly about traditional main stream media. In 2002 I created KDPaine & Partners and introduced the first low-cost measurement system in 2004. By about 2006 it was all about automated collection and analysis. And, of course, as social media exploded and the volume went thru the roof, so today you see mostly automated content analyses with human review.

What social media really means is that now media content analysis isn’t just for PR anymore but is seen as a way to listen to the voice of the customer (not just the media.) 

Of course the passage of The Barcelona Principles was a major milestone when PR people worldwide agreed on some basic principles of research and evaluation.

If we look carefully at a majority of entries submitted at international PR competitions, we often feel that "evaluation is good in general, but most organizations are still not doing it in the right way". How do you explain that even after Barcelona Declaration of Measurement Principles there is a lot of evidence that big players are still using the AVEs?


Agencies and vendors still use AVEs because they claim that “clients demand it.”  But in my 24 years in business, no client has ever demanded bad research or flawed data. What they think they want is one easy number. But what they really want is an accurate reflection of the value they add to the organization. What the Barcelona Principles established is that AVE is not that value. Because in truth, the value of PR can not be expressed in one universal number because value is based on goals and every PR program has slightly different goal. So there is no easy answer. It requires work and PR people are traditionally overworked, and there’s never enough time, so they resort to AVEs because vendors and agencies provide them. It’s only when they realize how flawed the data is – especially now with social media becoming so important – that they will ultimately reject them.

How should PR professionals approach research, measurement, and evaluation in social media?


Social media is really nothing new. People have been having conversations since before our ancestors came down from the trees. Cavemen grunting at each other and passing it along a trade route really aren’t that different than passing messages by courier or sending a telegraph or picking up a telephone or sending an email. The only thing that is different is that it is a lot easier (and probably more legal) to listen in on their conversation on Twitter or Facebook than it was to intercept a message from a guy on horseback.
So as always, in any communications program, you need to start with a goal. What problem are you trying to solve? What messages are you trying to communicate? 
Start with the stakeholders – what influences them? Where do they go for information? What are they seeing? What keeps them up at night?  Figure that out, and then you measure whether your messages (social or otherwise) are coming across.

What are the most important future challenges for the PR evaluation and measurement practice?


The biggest challenge we face is to measure what matters, not just what is easy. This means we must change the way we think about what it is we were hired to do. It is not enough to say that my goal is “to get on the BBC.” So what if you do? We need to ask the most important question which is “so what.” What does being on the BBC do for your organization?  And don’t just say “generate exposure” because the next question is also “so what.”  So what if 10 billion people see your brand, what does that accomplish?  And more importantly, what if 10 billion people see your brand and none of them get your messages. I’d be willing to bet that 10 billion people saw the News Corp brand in the last month, and very few of those “impressions” did the Murdoch’s much good. 


Katie Delahaye Paine is the founder of KDPaine & Partners LLC a New Hampshire- based research consultancy that provides measurement and accountability for corporations, non-profits and government agencies world wide. She is the author of the recently released Measure What Matters, Online Tools for Understanding Customers, Social Media, Engagement, and Key Relationships (Wiley, March 2011) as well as the popular text book, Measuring Public Relationships (KDPaine & Partners 2007). She is also the publisher of the first blog (KDPaine.blogs.com)  and the first newsletter (The Measurement Standard)  dedicated entirely to measurement and accountability. Prior to launching KDPaine & Partners in 2002, Paine was the founder and president of The Delahaye Group, which she sold to Medialink Worldwide, Inc. in 1999.

Katie was an initial founder of the Institute for Public Relations special commission on measurement and evaluation and is a Founding Fellow of the Society for New Communications Research. Katie was named one of PR Week’s 2008 Power Players for her advocacy of PR measurement. KDPaine & Partners won the prestigious Award of Excellent from the Society for New Communications Research in both 2009 and 2010. Her firm was also awarded the 2008 Jack Felton Golden Ruler of Measurement Certificate of Merit from the Institute for Public Relations. In 2006, Katie Paine received the Business Excellence Award for Excellence in Media & Marketing from New Hampshire Business Review. Paine has also been named Entrepreneurial Venture Creator, Person of the Year by the University of New Hampshire’s Whitmore School of Business. A Cum Laude graduate of Connecticut College's class of 1974, Katie majored in history and Asian studies. She received an Honorary Doctorate of Laws from New Hampshire College in May 1996. She is an Athena award winner and her life is featured in Mark Albion’s books, “Making a Life, Making a Living.” and “True to Yourself.”

For the past two decades, Paine has been providing marketers and communications professionals the tools, data and information to help them make better business decisions. She and her firms have read and analyzed millions of news articles, blogs, newsgroup postings and internal communications and have conducted hundreds of thousands of interviews in the relentless pursuit of quantitative and qualitative measures of her client’s marketing success. She works with some of the world’s most admired companies and organizations including Raytheon, Dell, Adobe and SAS. Most recently, her endeavors have been focused on social media measurement as well as providing cost effective measurement programs for non-profits, universities, small businesses and government agencies.


Interview by Dana Oancea, Forum for International Communications
Copyright PR Romania, all rights reserved