MONEYBALL: Statistics Makes Its Hollywood Premiere

January 20th, 2012

Imagine that you are a movie mogul looking for the next big hit. Two screenwriters come to you with a great idea for a baseball movie starring Brad Pitt. It combines America’s favorite pastime with one of Hollywood’s biggest names. It sounds like a sure hit. The next obvious question is, “What’s the storyline? “, to which the writers enthusiastically answer, “It’s about statistics!”

Though there is obviously more to the story, it is true that statistics plays a key role in the movie Moneyball . The film premiered this past fall and, though it has not become a major blockbuster, it has received some critical acclaim. Now, we aren’t movie reviewers. We’ll leave that to the experts. However, we are excited that a major motion picture shows the competitive advantage that can be had from statistical analysis (yes, we really ARE that nerdy).

Moneyball, adapted from a book by Michael Lewis, author of The Blind Side, is about General Manager Billy Beane and the Oakland Athletics. Beginning in 1997, Beane builds the club into one of the top American League Teams, despite the players being paid near the bottom of the pay scale. He does it by employing statistical analysis, or in this case, a type of analysis, called Sabermetrics. (For more information about Sabermetrics, see How Sabermetrics Works).

In a nutshell, Sabermetrics applies statistical analysis to objective data in order to calculate a team’s optimal performance. It’s the same principle that is used in determining the ROI of a marketing campaign (that’s why we are so giddy)…look at existing data, determine those elements that are most valuable and predictive, build the campaign and, “swing for the bleachers.”

Tags:
Posted in Data Mining, Optimization, ROMI | No Comments »

Do Your Customers “Belong” To Your Brand?

December 14th, 2011

One of the great things about the holidays is that almost everyone celebrates them. We come together to celebrate Christmas, Hanukkah, Kwanzaa or other traditions. By participating in the celebration of one of these traditions, we enjoy a unique spirit of community and togetherness.

It’s enough to make even a hard-boiled data analytics jockey like me stop and reflect.

What if a brand could create that sense of community and togetherness with its customers? How might that impact satisfaction, loyalty and repeat sales?

The conventional wisdom behind marketing’s entrance into social media is to encourage customers to become more fully engaged with a brand…to create a community of loyal customers. While it is still unclear how, and if, social media achieves these goals, it’s clear that the desire to “belong” to a community is significant. Maslow puts the need to belong just above our basic needs for food/shelter and safety.

So the next logical question is, “How do we motivate customers to want to belong to our brand community?” A common theme in many of our IdeaMap® studies is that customers want a greater level of control over their experience. This was confirmed in a recent study conducted by Pitney Bowes: “…consumers are more likely to stay engaged (with) companies that offer them a level of control over concrete customer service initiatives rather than those that focus solely on brand-building and web community experiences.” The study also found that an important way to encourage repeat business was to allow customers to have, “…a say in the company’s development of products and services,” (The Customer Dance: When to Lead, When to Follow).

As we celebrate the holidays this year, I think it might be worth pondering — “How can we create a greater sense of belonging to a community for our customers in the new year?”

Tags:
Posted in Optimization | No Comments »

A New Online Survey – How Confident Are You About Your Marketing Programs?

November 11th, 2011

We have tested thousands of marketing messages and campaigns over the last ten years. Why so many? Simply put, marketers want to have confidence that their marketing efforts will be successful.

In the past three years, confidence has been hard to come by, particularly when it comes to marketing. While the demand for proven marketing efforts has increased, the corresponding resources aren’t always available… making confidence difficult to achieve.

In an effort to determine the areas of greatest (and least) marketing confidence, Optimization Group has launched an independent online survey. Marketers can answer a series of short questions about their confidence in specific marketing techniques and tools. The results of the survey will be presented in a marketing industry report, which will be made available free of charge.

If you are a marketer interested in participating in the survey, you can CLICK HERE. The survey takes about 10 minutes to complete. Please be assured that this survey is confidential. Your participation will not be used for any marketing purposes.

If you would like to receive a copy of the report, you can submit your email address at the end of the survey and copy of the report will be emailed to you. You can also request more information about the Optimization Group.

Posted in Optimization, Research | No Comments »

Mindtyping™: Increasing Response Through Message Optimization

October 4th, 2011

Earlier this week a client joked that she is working on her Black Belt in IdeaMap®, an online message optimization tool popular with our clients.

For those unfamiliar with IdeaMap®, it is an online tool rooted in decision based conjoint…or trade-off…analysis. It recognizes that when people make decisions they are “trading-off” bundles of attributes against each other and models the trade-offs they are willing to make.

An IdeaMap® study will reveal naturally occurring groups of consumers – groups who are motivated by the same messages. We may find one segment motivated by emotional benefits and another segment motivated by facts and figures. We call this MindTyping™…knowing which segment a consumer is in.

At its inception IdeaMap® was limited to the effect on intent of each message individually.

Then came the Interactive Effects Analysis™, an analysis developed to identify the most powerful combinations of messages – maybe a feature + benefit or brand promise + brand name. Marketing communications are generally a mix of messages that tell a story, this analysis helps clients optimize the mix.

Not one to rest on our laurels…what about when a client wants to go beyond the optimal message to the target overall or key subgroups? These clients want to know the optimal message for Consumer A (maybe an emotional benefit) and the optimal message for Consumer B (maybe facts and figures).

In our work with the American Heart Association’s Go Red for Women campaign the AHA increased donations by 42.5% when they were able to identify which segment a consumer was in, then deliver a segment specific message to them. It’s all about getting the right message to the right person at the right time.

MindTyping™ represents the next evolution of IdeaMap®.

Maybe we will have to start referring to this new IdeaMap® capability as Black Belt Plus!

Posted in Uncategorized | No Comments »

Getting Unstuck: Making Smart Marketing Decisions When Data is Scarce

September 28th, 2011

Precise response data like leads, click-through rates, conversions and sales is valuable in making decisions about future marketing efforts.

But what do you do to if you don’t have data? How do you make marketing decisions when you have only partial information or just estimates?

That’s when a “What if…” analysis can be extremely useful.

A “What if…” analysis uses existing data, along with estimated data points, to calculate a range of probable outcomes. This information allows a marketer to make more informed decisions.

For example, suppose you are planning a new product launch and need to project the profit margin for the product. You know what the fixed costs will be, but variable costs depend on several factors that are not defined. “What if…” analysis takes the known factors (in this case, fixed price) and estimates of unknown factors (variable costs such as raw material costs, product pricing and sales) and determines a range of probable profit margins. This information can be immensely helpful in deciding if a new product will likely provide enough margin to justify further investment.

“What if…” analysis has been used for years by many major corporations such as General Motors, Proctor and Gamble and Eli Lilly.

While a “What if…” analysis doesn’t predict a specific outcome, it does provide a precise range of probable outcomes. By varying the input data, decision makers can see how those changes impact the probability of desired outcomes.

Tags:
Posted in Data Mining | No Comments »

A Solution to Measuring Integrated Marketing

September 16th, 2011

Over the past decade, the complexity and sophistication of the marketing function has grown by leaps and bounds. And, the pressure on the Chief Marketing Officer has never been greater – a recent study conducted by Spencer Stuart reported that the average tenure of CMOs at the top 100 branded companies is just 22.9 months compared to CEOs who are in their positions for an average of 53.8 months.

Today, more than ever, marketers are placing greater emphasis on integrated campaigns that leverage traditional off-line media, on-line activities, public relations, promotions, “social”, placed-based media, and now — emerging mobile applications. (No doubt, the list will continue to grow!)

The myriad set of marketing objectives and investment choices demands a new approach to measurement. The old decentralized approach relied on each marketing function or vendor to deliver relevant metrics typically used to justify maintaining or increasing the respective line-item budget. Stacks of independently created spreadsheets or reports were reviewed, but making solid fact-based investment decisions remained a struggle. Each activity had different measures: impressions, open-rates, re-tweets, conversions … some measures with direct attribution to sales and others without.

Our solution: “Follow the money.”

Simply, we integrate the disparate data into an integrated analysis-oriented database that normalizes resource inputs as dollars. For example, that “it doesn’t cost us a thing” social media campaign may consume 3.8 full-time equivalent staffers at an average fully-loaded salary basis of $xx,xxx per person. We do the math and end up with a standardized measure of inputs and outputs in dollars. We then use a flexible suite of data mining and modeling tools to identify the statistical relationships between all of the marketing activities … individually and in combinations … and the sales that result. A Matrix Oriented Analytical Approach (MOAA) identifies how various marketing activities impact sales across channels, across customer segments, across product lines. This insight enables you to make investment decisions holistically – based on total effectiveness to the organization.

Finding, collecting and organizing the data can appear to be a daunting task. And quite frankly, it can often be tedious and time consuming. But after all, “It’s just work.” And the combination of our tools, processes and partnership with IMTS provides a tremendous value in getting this critical step completed.

Senior executives with both client marketing and advertising agency management experience direct the Return-on-Marketing-Investment (ROMI) analyses, including development of “What-if Simulators” and dashboards when appropriate.

Cost and timing parameters vary. The key drivers are the amount and condition of data, and the number of products or channels to include in your analysis. The best way to get started is to conduct a data audit. If we move forward with ROMI project, the $4,500 fee will be applied to the project. If we find that a ROMI project is not possible for some reason, we will document our data audit findings and provide recommendations so that you will be ready to conduct one in the future.

Tags:
Posted in Optimization, ROMI | No Comments »

Revenue Performance Management (RPM) – The Next Big Thing, or Just another TLA?

April 29th, 2011

I was recently asked by Analyst Lauren Carlson to comment on a blog post discussing the new concept of Revenue Performance Management (RPM). As stated there and elsewhere:

“RPM is a systematic approach to identifying the drivers and impediments to revenue, rigorously measuring them, and then pulling the economic levers that will optimize top line growth.”
Brian Kardon

A couple of companies have created systems to support RPM by making it easier to gather, compile and analyze metrics supporting the concept:

“By pulling data from traditional CRM systems, RPM solutions allow users to gain a more holistic view of the revenue cycle from the earliest stages of marketing through to sales execution.”
Lauren Carlson

RPM’s advocates position the concept to be a step beyond traditional Marketing Automation, in that it concentrates on driving the top line and thereby captures interest in the C-suite. It may also be a worthwhile tool for those who want to compute their Return On Marketing Investments (ROMI). This utility alone makes it attractive to anyone wanting to hold marketers accountable for their results. We know that this issue is a top concern for marketers, as noted in the Association of National Advertisers 2011 survey results.

Measuring the effect of revenue enhancement activities is a natural extension of the data-driven business process improvement mindset that has become popular since Dr. Edwards Deming’s ideas were finally accepted in the US. The concept meshes nicely with standard TQM practices. Taking advantage of the wealth of data available in a well-used CRM system, plus other measures collected by the users elsewhere, RPM can enhance productivity, help make marketing a more efficient user of resources, and reduce uncertainty.

RPM is not a cure-all though. Reaching the right audience at the right time with the right message is still the essence of effective marketing. Those are challenges that must be answered outside of any RPM platform.

Posted in ROMI | No Comments »

Should Social Media Replace Surveys?

April 18th, 2011

Recently, there has been much discussion that social media may drive down the use of surveys and focus groups. Some have claimed that surveys could become obsolete in the next 20 years. That’s an interesting concept, but some of the talk may be hyperbole designed to attract an audience. One thing is certain, though: Social Media cannot be ignored.

Platforms like FaceBook, Twitter, Company Websites, Blogs and Forums all provide a rich, valuable source of customer feedback. Unlike more traditional MR data sources, the Social Media participants are running the show. They decide what’s important enough to them to talk about, to respond to, and to debate.

Dipping into this sea of commentary is the digital equivalent of eavesdropping at a cocktail party – you’re almost certain to hear something juicy, but you better have a sense of the context before you go repeating it!

And therein is the challenge: How does one get the most out of the Social Media data stream? How can we separate the signal from the noise? How do we define and identify a trend? What can we assume, or not assume, about the respondent demographics to know if they speak for our market, or for a segment thereof?

This Text Analytics challenge is the focus of many in the MR community. It’s an exciting and formidable frontier, because unlike the numbers of mathematics and statistics, all languages do not obey the same rules of grammar and syntax that we must use to parse out the meanings from the prose. Researchers are already using Text Analytics on SM sources with some success in discovering evolving trends, finding potential PR issues and collecting some crowdsourced opinions. Some are basing their work on an extension of the tools used for Open Ended Coding, while others are developing entirely new approaches.

So far, the toolbox seems to be incomplete. We’re not at the stage where we can send in a bot and pull out all of the goodies. Some of the hurdles are large, but the minds working on this are intelligent and dedicated. It’s a big task: Any tool must have at least language-specific versions, with their attendant cost and complexity. And, it must go far beyond word counts – what does the number of mentions matter if the context is unknown? Even “simple” sentiment detection is not so easy; English alone has dozens of different constructions for expressing positives and negatives, some of which depend on word order alone. Whoever gets this right, or close to right, will get us closer to true AI than MR has ever been.

The survey isn’t going away, though. While its use will reduced as our ability to mine the Social Media data grows, we will still find many applications where the traditional survey remains the best available option.

Tags:
Posted in Social Media | 4 Comments »

“Is that result statistically significant?”

April 18th, 2011

A simple question from a client at a recent research presentation. And I’m sure they expected a simple Yes/No answer. But (as you may guess), the right answer is a bit more involved.

While the theory behind significance testing is simple and elegant, textbook definitions tend to cloud the concept with opaque jargon: e.g. “the calculation of the acceptance/rejection region surrounding the null/alternative hypotheses.” Potential misunderstanding is furthered through a set of conventions that are often codified as default settings in statistical software packages. Most of us have been trained to look for “magic values” of α=.05 (95% Significance Level) or α=.01 (99% Significance Level) … and we deem anything else to be not significant.

But what does statistical significance really mean?

First the theoretical definition: A α=.05 means that the probability of erroneously rejecting the Null Hypothesis due to random error is 5%. This type of error, also known as a false positive, is a Type I error. Since there is a Type I error, you may correctly infer that there is also a Type II error (β-error) – which is the probability of accepting the Null Hypothesis when it should have been rejected (also known as a false negative). Both types of errors can be problematic and at a given sample size, reducing one type of error generally results in increasing the other type. By the way, the only way to decrease both types of errors is to increase the sample size – which is often not feasible or can be cost prohibitive.

Now let’s look at what a Type I and Type II error means from a decision maker’s point-of-view. Think of a Type I error as “an error of commission.” The decision maker concludes and acts on information, when in fact he shouldn’t have. And a Type II error is “an error of omission.” The decision maker concluded that the evidence wasn’t compelling enough to act … and did nothing.

Which error is worse?

It depends on the situation. If you are in pharma and deciding whether or not to introduce a new compound, you’d better control for Type I errors – you want to be 99.9999+% sure that you aren’t introducing something that will cause harm. And you’re willing to leave a potentially helpful (and profitable) drug in the lab until you’re sure. An error of omission (NOT introducing the new product) will generally be preferable to making an error of commission (introducing a harmful drug).

But if you are an Advertising Director deciding whether or not to launch a new campaign, you might want to balance the two types of errors differently. If you make a Type I error, you make an error of commission: you might invest $5MM in a new campaign, and not end up with much to show for it. The downside? You lost $5MM. But if you made a Type II error, an error of omission … you would NOT introduce the campaign when, in fact, you should have. And, the downside? It might be priceless. What if that $5MM campaign would work and generate $100MM in incremental sales?

So how do you balance the errors? How do you set the appropriate α-level? Don’t accept standard conventions — think deeply about the risks associated with each type of error in the context of the decision you are making. How confident do you need to be to take action? What’s really at risk? And what are the risks of NOT taking action – what are the opportunity costs associated with doing nothing? We suggest you explicitly incorporate this conversation into every project planning session. The following steps should help:

  1. First, assess the risk, here defined as the costs of erroneous action or the lost opportunity costs of non-action
  2. Then, design the project so that the error types are balanced in accordance with the placement of the risk. Since the risk is known, or at least better understood, decisions like whether or not to spend incremental dollars for a larger sample can then be evaluated in context. (Basically buying peace of mind, the price of which is likely related to the degree of risk)
  3. The project’s results are then evaluated more or less at face value, since it’s already been decided that the designed significance level is sufficient based on the contextual risk.

Following these steps will guide the decision parameters to fit the risk profile for the decision being made.

Tags:
Posted in Research | No Comments »

Marketing Research vs. Marketing Analytics: What’s the Difference?

March 22nd, 2011

This month’s Harvard Business Review features an interesting article on the value of conducting business experiments (A Step-by-Step Guide to Smart Business Experiments). It’s a topic that hits close to home for research and analytics firms like us. Be it experimentation or analytics, the objectives are the same — “to gather accurate data, analyze it for insights, and use those insights to make better decisions.”

That’s a decent mission statement for market research.

Analytics is a wonderful tool when you have access to a lot of data, the resources to organize and analyze it, and the skill to communicate and exploit the results. It’s a highly technical discipline that can yield useful results. But, in our experience, comparatively few small to mid-sized companies can pull that off consistently.

Experimentation is an excellent alternative to analytics. Rather than focusing on the past by analyzing historical data, the experiments happen in real time, with real customers and often with immediate feedback. At the end, as with analytics, you will have data instead of intuition to guide your decision making.

Experimental designs can be quite basic. All that you need are test and control groups, plus a way to measure all the results. Then, it’s a matter of doing one thing with the test group, another thing (or nothing) with the control group, and comparing the results. Retailers, advertising agencies, and direct marketers have relied on experiments for decades, using them to set prices, optimize product mixes, test copy, and to determine size and type of offers.

Whether you rely on experiments, analytics, or both to feed your decision making process, you’re ahead of those who don’t.

Posted in Optimization | No Comments »

« Older Entries