Are you designing the wrong thing?
I’ve worked on some seriously massive failures in my time.
The things I consider massive failures haven’t failed because of the usual things people worry about - being late, looking shoddy, being a bit slow, being unstable, lacking finesse etc. In fact, I've never worked on anything that's failed for these reasons.
They failed because nobody used them.
The countless microsites for FMCG brands with huge ambitions and tiny budgets, the part-time entrepreneurs that tell you ‘nothing like this exists in the market today’. They didn’t fail because their intrinsic quality wasn't high enough, they failed because nobody wanted or needed them.
Is there a need? How big is the need? Of course, you can never be truly certain, but you can be empirical about it by validating your decisions up front.
Now I know this sounds radical, but a nice way to test this is just to ask people. About 8 people is fine to begin with, then (if you have the time and money, validate your findings with a larger number).
Oddly though, agencies are really very bad at this. The usual excuses tend to be “There isn’t enough time”, “It’s too expensive”, “Users don’t know what they want” or “You should know what they want, you’re the designers!”. The trouble mostly lies in the way clients tend to engage their agencies. The agency pitches work that looks good, the client buys it. The agency is happy if the client is happy - no sense rocking the boat by seeing if anyone actually wants this thing. And besides, whose budget is the testing going to come out of? If the agency pays for validating it’s idea, the agency knows that the chances are, it’ll get canned. The truth is that, in the long run, you can’t afford not to consult with users up front because the cost of designing the wrong thing is astronomical in comparison. And it’s not just about the big ideas, it’s the smaller features too.
So, a cheap(ish) way of testing concepts up front is using a thing we've dubbed Pub Propositions (or alternatively The Mum Pitch). Find 8 - 10 people who are the kind of people you hope to use the thing. It will take a few days to prepare and it will cost you whatever you charge for your time plus the recruitment cost and the incentive for participants.
Pub propositions
A pub proposition is a sentence, which you can imagine saying to your friend, that describes in simple terms The Thing You Want To Make. Make it as simple and as realistic as possible. Don't think about how the thing is being marketed, just the benefits to the listener. It has to be something you can imagine saying in the pub or to your mum.
So, “It’s a website where you can type in what you’re looking for and it’ll give you a list of places to find it” is a good way to describe a search engine. You can then add a load more detail about how it actually works, because that’s usually the question people ask next. Adding a little sketch can help but you need to be careful that it actually aids understanding, not hinders it.
I’ve found that you can test about 15 pub propositions in 45 mins before people start to jumping out of windows. A group of about 8 of your target market will give you enough information to work with. Once you’ve talked them through the ideas, get them to rate them from 1 to 5. This works well for individual big ideas but for individual features you might want to categorise them more into “Don’t like”, “Bread & Butter”, “Nice but I wouldn’t use it all the time” and “Love this”. You can learn a lot about the way people talk about these things in response.
8 comments
The great thing about working in this way is that you can also quickly iterate on your pub propositions. You may actually have a good idea that you’ve simply expressed badly, or a small pivot on the idea can be shaped into a new and better pub proposition and tested again very quickly, and therefore relatively cheaply.
You forgot to do the Henry Ford quote about faster horses. That’s almost guaranteed when you bring up testing.
While I love pub propositions, I’m looking for the word insight. The connections we make from taking data and constructing what if statements is critical front end work in design and innovation. That way we are prepared for the ‘why’ of the client or why in the pub. Being able to say because and follow with a clear insight that is instantly gettable and repeatable gives foundation to any invention or new product in an existing category. See, what people say after because is the insight. It can’t be made up. It has to be drawn from data. (captures behaviour) and set up by an if then statement. The best example from 2010 was Jane McGonigal at TED. She put forward some data (By the age of 21, the average Americain will have played 10k hours of video games) and then put forward a theory: the 10k hours makes them experts at problem solving and then delivered 4 critical insights that follow because: games create meaning, games create tight social groups, games create blissful productivity and games create urgent optimism. Now any games inventor can use that work to explain their work. Unfortunately, whatever we design or build or invent has to be human and relevant. How? We find design strategies help. Critically though, if you don’t get a good reaction in the pub from an actual person who you’d like to appeal to, you’re right: you’re barking up the wrong tree. That’s why we like prototyping. Brings it all to life
James, oh god I hate that one, I’m saving that for another post – “Top 5 excuses for not consulting users”. The pivoting thing is ace, it’s amazing how much a subtle change in wording can take something from disgust to joy.
Martin, fascinating stuff. Though I often worry that insights are abused for the purpose of post-rationalisation and selling.
@thepaperbicycle: Surely these pub propositions are yielding the insights you seek – as early as humanly possible. If you were to get a negative reaction at this stage – i.e. way before you’d even finished developing the idea – that’s going to save a lot of time, budget and effort. There’s a tendency in traditional thinking to fetishize ‘insights’, which are treated like some kind of magic incantations. Ironically, the classic approach to insight-making is often linear, slow and can easily abstract the problem/solution to the extent that it inhibits useful and actionable direction. I see the pub insight as a particularly excellent form of prototyping: one that involves beer and mates. It costs little, you get a lot out of it, and you can fit it into a pint. Awesome. I would love to hear more about your approach to insights. I may have completely got the wrong ed of the stick.
Some great and frankly straightforward thinking here. Starting with the fact that we all hide our heads in shame at similar massive failures yet keep on committing these acts.
What irks me most is how often people on this side of the water reject pub proposition research because they feel the sample size is too small. Many usability experts that I have spoken to have told me that you rarely need more than five participants to get a good idea of major issues. But when “advertising” folks get involved there is an immediate need for the comfort of large numbers and, , focus groups.
Also was looking for additional description of the intriguingly named “Mum Pitch” – will that be a later post too?
Not to steal Mike’s thunder, in case he is planning a post on the ‘Mum Pitch’, but to quickly answer Dan’s question…
If you can explain it to your mum (especially if your mum struggles to understand the difference between a telly and a computer, because they’re both basically screens, aren’t they?), then you can explain it to anyone.
And if your pitch solicits more than a ’That’s nice, dear.‘, then you’re on to a winner.
what about things you don’t know you need or want yet (which is effectively the premise of advertising)?
I didn’t think I would need an iPad (and I’m still not sure), but there’s millions of folk out there who felt the urge to go and get one. Contrast that with all the totally failed iPad imitators, who have something pretty similar, but not the marketing spend behind them to push this into our realm of consciousness.
Hi Robin, not entirely sure I get what you mean. If you took 15 Apple fanboys and asked them each if they would like ‘A touchscreen tablet from Apple’ 2 years ago they would be having wet dreams about it for weeks.