‘One big focus group’

I’ve heard this term thrown around a bit (Google it) to describe ‘naturally occurring’ conversations on the internet as a rich source of customer insight. It annoys me.

Listening to/gathering online content is absolutely nothing like a focus group.

Beyond the obvious (ie there’s not a great deal of focus in terms of sample, or being able to probe specific topics), and maybe surprisingly, it lacks some key aspects of spontaneity than can be generated in a focus group. Moreover, unlike a focus group, it doesn’t readily provide a good foundation for analysis.

Highly edited

In many cases that we, as market researchers, will be interested in, the content that ends up in the netographer’s dragnet – eg a blog post, a comment on a community thread, a tweet, etc – is the product of a process that involves considerable cognitive effort on the part of the creator; either word crafting a thought/response, choosing an image, shooting and editing a video etc.

And various factors will shape that effort, such as the intended or perceived audience, the perceived importance of that audience, the background and motivation for generating and posting the content in the first place etc.

Clearly, this is a highly controlled, highly edited process – a fact that seems at odds with the perception of unsolicited online content as somehow more authentic than the content a researcher can get via other (more direct) methods of inquiry.

But the key issue here is that there are very few clues to help the researcher understand the motivations of any particular individual driving their broadcast to the online world.

The mother of issues

As a qualitative researcher, motivation is one of the key factors I’m interested in understanding. It’s the very cornerstone of my analysis; the context. But I can’t readily get that online. Without directly asking, in a timely and appropriate fashion (a whole other blog post), I’m not privy to the backroom.

Of course, not understanding the motivations/context behind the content is fine if you’re simply gathering and presenting content. But it’s the mother of issues if you want to provide your client with any substance.

We’re getting good at capturing data and making it look pretty. But is our industry paying enough attention to its analysis?

Fluffy techniques

Useful qualitative research output has two defining features; depth and clarity. And perhaps surprisingly, this is where what might be perceived to be ‘fluffy’ research techniques come into their own. They’re an efficient way to get depth and clarity.

An example, asking the same question in two different ways, will help to explain what I mean.

Q: Does Brand X have an image problem?

There are 3 possible answers to this question; ‘Yes’, ‘No’ or ‘I don’t know’. Some clarity yes, but not much depth there. None of these answers are very helpful in terms of understanding anything about the possible image problem.

Let’s ask the same question, but in another way:

Q: If Brand X came to life as a person, what kind of person would they be? What kind of music would they listen to? How would they take their coffee? etc.

I can understand that to someone who hasn’t had much experience with qualitative research, the above line of questioning may seem somewhat fluffy.

Allow me to de-fluff it.

The output

As noted, the second question is just another version of the first. But the answers will be quite different.

Asking the question using this ‘fluffy’ approach delivers answers in 3D. And then following up, down and sideways on the answers with ‘Why? Why? Why?’ (not literally, in essence) helps to build a rich, relatively holistic picture of Brand X’s image. Critically, it also provides a context for interpretation.

The above technique is called personification. It’s just one example of a fluffy technique: there are many more in the qualitative toolbox. When used appropriately, fluffy techniques really deliver the goods. Research participants love them: they’re fun and engaging and something a bit different. Researchers love them because they enable deeper understanding of attitudes and perceptions, and thus greater insight.

Effectual fluff. Neat huh?

(This was a post I originally wrote for Marketing Mag)

Here’s another excerpt from my Marketing Magazine series on Qualitative Research…

Within a qualitative research context, there’s no right way to ask a question per se. There are actually many right ways to ask a question. And there are also many wrong ways to ask a question.

The wrong ways

You may have heard about some of the following heinous qualitative research crimes:

• Asking leading questions
• Asking closed ended questions
• Asking vague questions

Why are these ‘wrong’?

Because leading questions ‘lead’ people to a particular answer, closed ended questions can end the discussion prematurely, and vague questions elicit vague answers that have little grounding for interpretation.

Well, theoretically. But all is not what it seems. An experienced moderator might use any of these types of questions purposefully, and with excellent effect:

• A leading question often works well to test a hypothesis, or as stimulus in itself, to get the conversation going
• A closed ended, or vague question can provide a foundation to open the discussion in interesting and new ways

They’re all part of the qualitative researcher’s toolkit and used in a timely and purposeful way, can add tremendous depth to the discussion.

Here’s a post I wrote for Marketing Magazine about the ROI on qualitative research;

The value of market research – whether we’re talking about qualitative or quantitative research – is difficult, if not impossible, to quantify.

That’s because market research rarely, if ever, works alone in shaping strategy. It’s just one of many tools in a marketer’s tool bag.

In addition to this, market research is only ever:

• As good as the research brief and the questions it asks
• As good as the analysis and the debrief
• As useful as its end users make it; it’s what they do with the output that can determine success or otherwise

Given the variables listed above (so called because they vary), it’s pretty much impossible to put a figure on its value per se.

Relevance

Let’s look at it in another way.

If a particular product or service or piece of communication is relevant, it’s far more likely to end up in the shopping basket (so to speak). So the absolutely fundamental, most basic question for marketers should be:

“How can we make our products/services/communications more relevant to our customers/potential customers?”

And there are two ways marketers can go about answering this question:

1. They can ask their customers/potential customers
2. They can guess

Ask them

If marketers ask their customers/potential customers (and listen to them), they’ll be in an excellent position to create relevant products, services, communications etc.

The value of qualitative research here is obvious; it’s a very good way of asking, and listening, to your customers/potential customers to find out what’s relevant to them.

By being relevant, you’re optimising the chance of collecting the sale. Therein lies the return on your investment.

Guess work

If marketers decide not to ask and, in effect, guess what the market wants, they run the risk of getting it wrong.

Consider the time, resources and money wasted when bad guesswork delivers a dud. Go one step further; cost it out. And add the opportunity cost.

When you have that figure, my work here is done. Because that figure gives you a very good estimate of the ROI for good qualitative research.

Nothing to sneeze at, is it?

If a company offers products and services that have personal relevance, well, the cat’s half in the bag.

If they can work out how to tell me about those relevant products and services, in a relevant way, at a relevant time, then they’ve got themselves the full cat deal.

But what is relevant?

Hooray for qualitative research!

Following on from my last post…

To state the obvious, we don’t go into concept testing with finished ads. We go in with stimulus that (hopefully!) helps to describe the creative concept; maybe a script, a story-board, a mood board etc.

This is an important point; the stimulus is a description of the concept. For all intents and purposes, the description is not our focal point of inquiry; the concept is.

But while our focus is on the concept, it’s almost impossible to get responses that aren’t at least shaped by, if not entirely based on, the stimulus.

The problem arises when the researcher fails to identify, and disentangle, responses to the stimulus from responses to the actual idea. This means that negative responses to the stimulus are treated as negative responses to the concept.

The baby, the bathwater…all out the window.

Yes, a concept may need work. Indeed, it may need a total rethink. But if the researcher tries (and hangs) the wrong man, a concept with promise may never even see its day in court.

I’ve been busy. Nice, but not much time to spend on the ol’ bloggery this week.

So to keep some semblance of blog-momentum, here’s one I baked earlier for  MarketingMag.

(Reprinted here so that I get to use the Daffy image!  It’s all about substance of course).

How to choose a qualitative research consultant

So let’s assume you’ve been following my series in Marketingmag, have fallen in love with qualitative research, have decided to undertake a qualitative research project, and are now trying to decide on a qualitative research supplier.

What should you look for? Good question! And you’ve come to the right place; that’s what this post is all about.

The essentials

There are a few basic, but essential things you should look for when deciding on engaging a qualitative research consultant. Broadly speaking, look for experience, a good understanding of the role of research, and a passion for sample;

Experience

I’ve said it before and I’ll say it again; make sure only senior consultants work on your qualitative research projects.

Good qualitative research consultants have lots of experience; real world, real client, real respondent and real marketing experience. Experience means they can provide significant efficiencies in the research process and much greater efficacy in extracting insights.

Specific industry experience can be helpful, but isn’t always necessary. While it can provide focus more quickly, there’s also great value in having a fresh pair of eyes take a look at your market and issues.

Understanding

A good research consultant understands that they’re not the star of the show. They get that there’s more to the picture than the research per se. They get that it’s actually about the client’s bottom line.

Passion

Sample is everything in qualitative research. Good research consultants are very fussy, if not passionate about sample definition and sample recruitment.

The finer points

Above I’ve briefly listed some of the essentials. Let’s now look at some of the finer points of distinction that will help you choose a good supplier. Four, to be precise;

They understand quantitative research

Yes, you read that right. Good qualitative research consultants understand quantitative research.

I don’t mean they necessarily understand it at a regression analysis or chi-square level (feeling dizzy now). I mean that they truly understand its value as part of the problem solving mix.

Communication

Good research consultants have to be good communicators.

What’s the point of discovering earth shattering insights if those insights then just sit lifeless – all squished up in a rabble of page-cluttering bullet points? Yawn.

Good research consultants take pains to communicate their research findings in a way that gives them a useful and productive life.

Frameworks

I’m strongly opposed to using frameworks based on psychological theories that have little empirical evidence to support them (Maslow anyone? Or just pick any personality theory).

Many of these theories have zero credibility; they give both research and psychology a bad name. In my opinion, anyone using them is a practicing quack.

They’re proprietary free

I’m not a great fan of black box techniques in qualitative market research either.  “Proprietary” frameworks, tools and techniques make my toes curl.

My main objection (to be sure, I have many) is that they force the shape of the research input and output in a cookie cutter style.

They get in the way of seeing and they get in the way of thinking. And seeing and thinking anew is what good qualitative research is really all about.

The crunch

In choosing a qualitative research consultant, make sure you have the essentials covered; look for experience, a good understanding of the role of research, and a passion for sample.

Beyond these essentials, look for consultants with a good understanding of quantitative research and an ability to bring the research findings to life in an engaging way. Finally, stay clear of consultants who want to sell you questionable frameworks; go with the good, original thinking every time! 

Are two minds always better than one? I don’t think so.

Take a six group qualitative research project. Two moderators are assigned to the project, each running three groups. Timings, diary clashes, etc, mean that they won’t get to attend many (if any) of each other’s groups.

When the groups are finished, the two moderators get together to discuss their three groups worth of insight. They then stitch the findings together, in a perfectly gestalt fashion, to deliver >six groups worth of insight.

Not!

It doesn’t work that way. The insight from three groups, even if you multiply it by two, is still just insight from three groups.

What’s missing, is the incremental learning: each moderator has only three – not six – sessions in which to evolve and refine their hypotheses. Effectively, you end up with three groups worth of insight, but the client is paying for six.

*Raises one eyebrow*

The only way two minds can be better than one in this scenario is if both moderators watch all of each other’s groups, or just one moderator conducts all the groups.

Just saying.

See here for a brief explanation of the Porcupine principle.

Julian Cole, of Adspace Pioneers fame (and Social Media Strategist extraordinaire over at The Population), has compiled a list of the top marketing blogs in Australia.

So it’s actually more like 100+ porcupine reads: each with its own clever blend of fine thinking and discourse. Value.

And Zebra Bites made the list! Currently sitting, stripey and pretty, at #96. Interestingly, Zebra Bites is one of only 3 market research blogs on the list. Even more interestingly, it looks like it’s the only qualitative market research blog listed…

: o

This is a qualitative make-over post. 

I’m going to do a ‘before and after’ on a sentence I pulled from a qualitative research report I was asked to read recently.

Here’s the before sentence: Of the 4 groups, 83% of people said they liked the design.

Now some of you reading will, at this point, know exactly what I’m talking about: You can go and play. For those who don’t please keep reading.

There are many things wrong with the above sentence being part of a qualitative research report. Here are 3 points to start:

1. It’s not robust

Percentages, in a qualitative context, are pretty much meaningless. While at first blush, a grand 83% looks pretty good, what does it really mean? It means that, assuming 4 groups of 8 participants, 26.6 research participants, screened to fit a particular profile, and willing to attend a particular research group, said they liked the design. That’s a very small, skewed sample: hardly robust and hardly worth reporting. But that’s only the beginning…

2. It’s not controlled

To get a good, clean read on any particular issue in a quantitative survey, the way the questions are ordered and the way they are asked is key. For all intents and purposes, and as much as possible, the survey should be administered in a controlled environment. Even rotating the order of questions is controlled. This ensures reliability (being able to replicate the findings) and therefore, some confidence in the results.

In contrast, to get a good read in a qualitative study (we don’t necessarily go for clean in qual), we need to dance around a bit. Cover the floor. A good qualitative facilitator will bounce around, jump ahead, reverse, turn corners, step to the side…you might even see a grand jeté.

="font-family:verdana;">The point here is that the context within which the question is asked, ie the discussion group, will vary wildly for each group. In effect, it will be confounded by all sorts of, well, confounding variables; not least, the discussion itself. The fact that 83% of people said they liked the design means absolutely zip without understanding the discussion that came before.

3. It’s open to misinterpretation

The third and most worrying point is the potential for misinterpretation. The most obvious here is making the assumption that 83% of people, per se, liked the design. And then using this ‘finding’ to make Big, Important and Expensive decisions, like changing the design.

Here’s the after sentence: Positive feedback for the design was based on factors X, Y and Z.

Note the glaring (and appropriate) lack of percentages?

A quick read. Fast turnaround. Need a topline by Wednesday.

I haven’t quantified this trend, but based on our sample of one (Zebra!), there seem to be a lot more of these types of projects around.

I put it down to two key reasons:

1. The nature of our clients’ subject matter (Web 4.0 anyone?). On this front, it’s all about faster, more efficient, leaner…

The more you know about something, the more you realise that you don’t really know very much about it at all. Relatively speaking.

This is a gem that’s held true for my whole learning life. It’s a concept I love, albeit somewhat reluctantly.

It means there’s always more to learn: a lovely thing for a qualitative researcher. On the other hand, it’s not quite so lovely for arriving at conclusions. 

Further research is (always) warranted…

Let’s start with some context and an example.

Some context

People buy things. People sell things. How can people who sell things get more people to buy (more of) the things they’re selling?

An example

Just say we want to sell cupcakes. A good first step is to try to understand a few things about cupcake buyers, or potential cupcake buyers:

• Why do people buy cupcakes? What need or desire do cupcakes fulfill?
• When and where (and why then and there) do they buy cupcakes?
• What is/who is involved in the cupcake decision making process?
• What’s the best way to communicate with cupcake buyers/potential buyers?

These are the kind of questions that a qualitative researcher would ask. Our objective is to understand attitudes and behaviours around cupcake buying (and eating!).

This information will help us develop appealing cupcakes, based on buyers’ needs and desires. It will help us develop relevant and appealing messages about our cupcakes. In effect, it will help us to sell more cupcakes.

Before I start talking about what qualitative researchers do, I thought it might be useful to cover some basic definitions. Specifically, what is qualitative research? How is it different to other kinds of research?

Very broadly speaking, there are two kinds of market research: quantitative and qualitative.

Quantitative market research is generally used to measure or quantify a given variable of interest, eg how big is a market, how many people hold a certain attitude or behave in a certain way, like a particular brand, bought product X, etc.

Qualitative market research, in comparison, is used to provide understanding. Here, the inquiry is focussed on why people hold certain attitudes, behave in a certain way, or like a particular brand or product over another, etc.

I recently got in touch with a long lost, very dear friend of mine. We met at uni many years ago and he now works in IT. He’s extra-clever: one of those people that ‘gets’ things. And he’s just a really, really nice guy. Top that!

But I digress.

What’s extraordinary – why I’m blogging about this – is that when I told him that I was a market researcher, mostly undertaking qualitative market research projects, I saw not just a hint, but a great wave of confusion cross his face.

So I paused.

“And what exactly is it that you do?” he asked.

“You know, find out what people think about things: products, services, brands…you know. And try to work out how to talk to and with them in a meaningful way…”

Didn’t cut it. He didn’t know. And if someone that smart and quick to get stuff didn’t get it, it’s got to be my fumbling explanation.

So over the next week or so, I’m going to try to explain just what it is I (ie as a qualitative market researcher) do. I’m going to try to make it easy to understand for someone new to, or not in, our industry.

: o