I’ve heard a few times now – from some within (and some flirting on the edges of) our industry – that market researchers need to stop focusing on research methodology so much.

There are two main ‘reasons’ put forward here;

1. First, that methodological considerations are so elementary that we don’t need to talk about them any more [they’re not, and we do]
2. And second, that it’s simply passé to focus on methodology; if we do, the market research industry will be left behind [it’s not, and if that’s the reason the market research industry is going to be left behind – if, indeed, it isactually going to be left behind – I’ll eat my stripy hat].

The upshot of these discussions is the irritating and ill-defined – but always emphatic – argument that we, as an industry, need to innovate.

But sound methodologies and innovation are not mutually exclusive concepts.

Notwithstanding that research methodology, itself, can be innovative, we need not, should not, and must not sidestep careful methodological considerations for each and every market research project we undertake.

Careful methodological considerations – built into the research design, and used to frame the analysis – are a fundamental bedrock for useful innovation.

Do you agree or disagree with the following statement…?

Data quality is critically compromised when double-barreled questions/statements are used in market research surveys and researchers who write them into their surveys should be shot.

;P

Another question from the discussion.

Q3: Is consumer attitude shifting in regard to research participation?

A: Chinese whispers from quant-land suggest that long, boring surveys are failing to engage respondents. This isn’t new, but there’s more research on research now, and thus we’re hearing more about it.

I’ve no doubt that poorly designed surveys would spark a shift in terms of how many people are willing to give their time to participate in research, and the price they set for their contribution.

Things are a bit different on my side of the research fence; ie qualitative.

In this context, buzz words such as ‘collaboration’ and ‘co-creation’ are being thrown around as the new terms of research participant engagement; these concepts underpin the fundamental premise of online market research communities.

But actually, collaboration and co-creation aren’t new concepts in qualitative research. In fact they constitute the very essence of most qualitative research – on or offline.

It’s always been a dialogue between researcher and respondent/participant. We ask questions and they collaborate in answering. Or they ask questions and we listen and ask questions about the questions…etc. We show them stimulus and they co-create a vision.

Beautiful.

Following on from my last post, here’s the second question and answer…

Q2. Are there any lingering misconceptions that marketers attach to market research?

A: Four misconceptions de jour;

1. That online research is quicker and cheaper. It usually isn’t if it’s done well. The technology doesn’t provide the value. It’s the analysis that provides the value. And that’s what takes thinking and time (note, I can only really speak for qualitative here).
2. That numbers (quantitative research) are more important than the sentiment around those numbers (qualitative research). They’re both important.
3. That market research is a cost. Good research is an investment in managing risk.
4. That Maslow provides a good framework for interpreting research results. It doesn’t.

I recently answered a few questions about market research for Marketing Magazine. I must say, it was nice to be at the other end of the questions!

I think the article only appeared in the print edition (June, 2009), so over the next few days I’m going to post the nutshell version of the questions (4 altogether) and my answers.

Please feel free to add your own thoughts!

Q1: How do marketers get the best value out of market research when their budgets are being diminished?

A: Marketers will get the best value out of market research by making sure they do 3 things;

1. Ask good questions (ie have clear and well defined objectives)
2. Ask the right people the good questions (ie employ a useful sample)
3. Ask the right people the good questions in the right forum (ie choose the optimal methodology)

Sounds simple, but this is where the clever thinking is. If you get these 3 right, you’ll get useful data, efficiently.

I’ve always been a big fan of desk research and in particular, the literature review.

Even in the days when it meant searching the CD-Rom database at the UNSW library (any one remember Psych-Lit?) to find the ‘literature’.

I’d scribble down the references on a scrap of paper and go, on foot, to hunt for the hard copies. These would invariably be housed up on the fifth floor, down the very end, where the heaters didn’t work in winter. But it didn’t diminish the thrill of the ‘find’.

I danced with (some might say nerd-like) delight when it became possible to access the library catalogue via the internet (albeit in the early days, you could still only get information on the item’s location, not the actual item – but it was still exciting!).

Things have changed considerably since then.

From the way we source information, to the type of information we end up with. All changed.

But importantly, it’s not just the content that’s changed, it’s probably also the quality.

Why? Because some information is (a lot!) easier to get than other information.

I sometimes wonder whether we’re too quick to stop at the easy-to-get-to stuff.

Do we know when and how to dig further for better information?

Are we teaching the next generation of researchers how to do this?

A skill worth honing

Being able to weave a literature review together is a skill worth learning and/or a skill worth honing, particularly for researchers.

Beyond the likelihood of increasing the quality of information one ends up with, the process can be enormously enlightening.

It provides a feel for the breadth, and often surprising depth, of knowledge around any particular subject; inspiring and humbling at the same time.

It provides exposure to confronting, yet compelling views, often contrary to one’s own.

It’s a way of learning what wheat looks like vs chaff.

More context. Better information. Better research.

Are you measuring what you think you’re measuring?

Busy times at Zebra, so just a snack-sized bite…

Question: What’s the difference between good market research and bad market research?

Answer: Good market research makes you money; bad market research costs you money.

I think a key part of our role, as professional market researchers, is to advise and steer our clients on, and towards, the research methodologies that will effectively and efficiently answer their research questions.

“Yeah, and…?” I hear you mutter. “You’re stating the bleeding obvious”.

If it is the bleeding obvious, then I’m confused.

Because while I busy myself with answering that brief, the passionate embrace of all things 2.0 (for the want of a better descriptor) by some researchers – along with the often alarming and dire warnings for the future of Research 1.0 (for the want of a better descriptor) – suggest that marketers must have suddenly changed their questions.

Have they?

In some cases, yes. The world itself has changed/is changing, and the marketing context is changing too. But it’s not changing entirely, and importantly, it’s not always changing in parallel.

A Research 2.0 solution suggests that marketers’ questions have changed as quickly and as radically as online technologies and forums themselves.

But marketers still want to know how to sell stuff. Does Research 2.0 output help them do that? Does the information gleaned from the ‘new’ listening posts answer their fundamental market research questions?

Or are those at the helm of the Research 2.0 front actually changing the brief itself? Shaping the research questions to fit the new technologies?

Is it research?

Perhaps what we’re defining as Research 2.0 isn’t, in fact, market research at all. Maybe it sits outside the realm of market research; more in the customer relations/customer service domain.

Without doubt, the online environment provides marketers with invaluable feedback – but of a very specific kind. Quite clearly, because of its skews and tip of the tailness, it’s not the kind that’s of much use in making big-marketing-budget-decisions.

Maybe, as market researchers, we are in pole position to harness and distill that specific feedback for what it may be worth. We can certainly lend our experience and caution to the analysis.

But are Research 2.0 methodologies really the silver bullet they’re being sold as? Is it a marriage of the right questions, with the right sample, in the right context?

Or is it a shotgun wedding?

Maybe I should stick to knitting, but despite my obvious leaning towards qualitative (vs quantitative) research, I feel compelled to write about a quantitative research issue that isn’t quite getting the consideration it should; sample representativeness. This, I might add, seems to be a problem particularly – although by no means exclusively – for quantitative research conducted in the online environment.

Sample representativeness

When we (when I say ‘we’, I don’t mean you or me of course, I mean them, but let’s go with ‘we’) undertake a quantitative market research study, it’s rare that we have either the time, or a budget, that would allow us talk to each and every customer/potential customer of interest.

Instead, we choose a selection of those customers/potential customers to represent the greater population of interest to us. In research-speak, this selection is called the sample.

In a quantitative research context, the way you choose your sample, and the structure of that sample, is everything. These two factors will pretty much define the extent to which you can extrapolate your research findings to the population of interest. In non-research speak, that means the extent to which you can have any confidence in the research results.

Making sure that the sample you want to use to generalise to the greater population of interest is representative of that greater population is;

1. Of vital importance

2. Not always easy

3. Of vital importance

Number 3 isn’t a typo. Issues arising from number 2 often mean that the sample may be seriously compromised. I put number 3 there as a reminder.

As you all probably know, I have a ‘thing’ for questions; love a good one, loathe a bad one.

Why? Because within a research context, asking good questions is absolutely, fundamentally, and without doubt, a prerequisite for getting good (ie useful) information.

And as we all try to acclimatise to the great unwashed world of Web 2.0 metrics and mumblings, it’s becoming increasingly obvious that asking good questions will be one of the greatest challenges – but will also return the greatest rewards – for marketers and market research consultants alike.

How sharp, how ‘good’ are your questions?

More on this to come…

I think I’ve found my blog-soul mate.

Nice to find someone who shares my passion for user friendly research.

*Just in case* you missed my rather excited tweets or Facebook update, here’s my opinion piece in this issue of Anthill magazine.

: P

The seedling is here.

There’s an interesting debate brewing over the definition and use of the term “Usability Testing” over at Patrick Kennedy’s (excellent) blog.

My (very possibly misguided) take out is that “Usability Testing”, in its proper form, is a specialised practice that often sits under the broad umbrella misnomer (drum roll) of “Usability Testing”. And when people use this term to describe other elements of usability testing (without the quotation marks), they actually mean usability evaluation.

Get it? Don’t worry if you don’t. It’s just some context, but not the point of this post.

The point of this post is a question; is the skill set of a User Experience (UX) professional different to that of a qualitative research professional? If yes, how?

Rhetorically yours (until someone sets me straight!)

; )

I’ve seen quite a few written research briefs in my time.

And most of them, from a researcher’s point of view, have been hard work. Too much information, not enough information, odd information, wrong information, etc.

This used to really bug me. Why, I’d wonder, tearing my hair out, couldn’t clients learn to write good research briefs?

And then, one day, it clicked. Humble pie, etc. It’s not really cricket to ask, or expect them to. A clients’ job is to use the research, not to have to design it too. Ahem. That’s my job.

While a beautifully written brief might make our lives, as researchers, easier, it won’t necessarily make the research any better. In fact, more often than not, the most useful information for developing a research programme comes from sitting down for a chat with a client. Talking through the issues invariably helps both parties shape and focus their thinking well beyond a written brief.