Are designers really better judges of design quality than users are?
Discussion inspired by the Ghost in the Pixel blog post Why designers don’t like A/B testing by Uday Gajendar.
The blog post was written in response to this tweet: http://twitter.com/#!/lauraklein/status/73135201416527873
I don't trust designers who don't want their designs a/b tested. They're not interested in knowing if they were wrong. #leanstartup
Gajendar proceeds to explain why it's basically impossible for designers to be wrong. Though he makes a couple of good points, I find myself unable to ignore the smell of designer arrogance, wafting up from the page in waves like the fumes of a beached whale decomposing on the beaches of Santa Cruz, California.
Probably the most irritating of Gajendar's claims is the following:
Users are not always the best judge of high quality design. That’s why you hire expert designers of seasoned skills, experience, judgment, and yes the conviction to make a call as to what’s better overall.
O RLY??? Are there any objective metrics by which a designer may be assessed, then? Or must I get a master's degree in HCI before I'm qualified to discuss possible weakness in your design (such as the fact that it's too complicated for users to understand)?
While I don't believe in A/B testing any more than Gajendar (at least not for startups), it really seems to me like his attitude can be summarized as follows: The designer is right. If the users don't get it, they're stupid and lack training in UI/UX. If your A/B testing suggests another design might be better, than you're a n00b for using A/B testing instead of listening to me -- the expert.
It seems that so many designers feel like they are accountable only to their own sense of Platonic design ideals, rather than to, say, the company and its mission. They often emphasize the importance of having a grand unified vision for the sites design, and breaking features down into conceptual categories that are probably more meaningful to the designer himself than to the average user.
And when it comes down to it, aren't you building the product for your users? Doesn't their opinion matter quite a lot? Granted, you don't succeed by listening to everything users say, but it's nonetheless crucial that you incorporate direct and indirect user feedback in all its forms: entries in the error logs, use (and misuse) of features, emails to support staff, etc.
Look at Quora as a case study. Listening to people "in the know" (i.e., UI/UX and designers in Silicon Valley) Quora is "like Yahoo! Answers, but better because it focuses more on UX." But when you let designers run wild, completely unaccountable to any kind of objective reality, the result is a poorly socialized product which does not play nicely with the average user.
I don't mean to knock either designers or Quora, but I do think that, just like everyone else, designers should remember that they are a member of a team, and that they are really there to help the team figure out how to make the product successful, not create a work of art.
Yes, we build products for our users.
A product is a living, growing, evolving thing. It starts with a vision, and then it evolves once its creators see how people are actually using it.
In practice, we never know how people will use a product until people actually use the product.
Despite the school of What would Steve Jobs do?, it's the users who make the product successful, not the designers.
See also: What is the ideal relationship between design and engineering in a startup?
It is the users who ultimately make the product useful, and it's why whoever's listening to what your users are saying has to be not just a good listener, but someone you trust to be a competent filter in order to be able to make sure the product being built is still one the users want.
Just my two cents, and that's probably all my feedback's worth :P
Isn't this a little bit like saying is doctor better judge of the illness the patient has instead of the patient who is actually ill.
You could look at it that way if you feel that designers know something that the rest of us do not.
I personally feel like it's similar to the a musician saying that they are a better judge of music quality than people who simply enjoy listening to music.
If most people agree that some musician's latest album is not good, do we necessarily trust that less than the musician's opinion (which is probably that the unwashed masses lack the refined palate to appreciate "truly good" music)?
But I'm personally of the opinion that UI/UX design is more akin to art than to science.
Evidence-based design is BOTH an art AND a science: http://www.uxmag.com/design/the-art-and-science-of-evidence-based-design
I prefer the words of Patrick Kennedy, who said, "The scientist and artist within all of us should be grappling with each other to keep each on the level."
Source: http://www.gurtle.com/ppov/2009/06/03/is-ux-an-art-or-a-science
I am ancient by all accounts - having studied UI/UX design in the *80's* (!?) at Cal, but I did happen upon an interesting insight (I think) while doing my 3rd start-up - an education company. After sitting through endless presentations on good learning design (blah blah blah "different learners = different learning styles" blah blah blah), I came to the belief that good UI/UX design shares a lot in common with good learning design: different users, different styles (better listen to A/B feedback at some point, lest you miss a constituency you don't understand); inspirational educators innovate in ways that are so far out-of-the-box that it is difficult to evaluate the technique until it is (nearly) fully formed and in use (supports the "god-designer" belief of no A/B testing); all good learning design incorporates continuous feedback (A/B testing); the only "complete" education material is on a topic that is no longer relevant... All UI Designers are at heart educators of software use patterns...
Feedback loops are powerful things, that can be employed both to create and to destroy:
"A feedback loop involves four distinct stages. First comes the data: A behavior must be measured, captured, and stored. This is the evidence stage. Second, the information must be relayed to the individual, not in the raw-data form in which it was captured but in a context that makes it emotionally resonant. This is the relevance stage. But even compelling information is useless if we don’t know what to make of it, so we need a third stage: consequence. The information must illuminate one or more paths ahead. And finally, the fourth stage: action. There must be a clear moment when the individual can recalibrate a behavior, make a choice, and act. Then that action is measured, and the feedback loop can run once more, every action stimulating new behaviors that inch us closer to our goals."
Source: http://www.wired.com/magazine/2011/06/ff_feedbackloop/all/1
@ifindkarma Actually, Adam, your comment hit the nail on the head. There is art in the balance. Designers dream. Users (in)validate.
You May Also Like:
