In the Methods of Human-Computer Interaction course that I took last Fall, we learned about a number of Discount Usability Methods, including Heuristic Evaluations, developed by Jakob Nielsen. The thing about HEs is that if you get enough people together, you can find a majority of basic flaws with any given interface. The training is really cheap, and the evaluation itself shouldn’t take too long either. In all, it’s pretty inexpensive, especially compared to some of the other methods out there (bringing in “real users” can get expensive, as I’m learning this summer).
As such, I was shocked to learn that you can commission Nielsen to do a Heuristic Evaluation of your website or interface for $35,000! That’s not very “discount,” is it? On the upside, you get experts from the Nielsen/Norman group conducting your evaluation. But still, according to the literature I’ve read in this program, a group of 5 Heuristic Evaluators will find about 80% of the issues present in an interface — not 100%, but is that extra 20% worth tens of thousands of dollars? It’s not that good of a method…
I have found that double-experts, much like double-cream brie, are usually worth the money.
I think that $35k isn’t all that bad. So, say that they put 3 people on the job, and that it costs twice as much to employ someone as they make in salary. And, being in the bay area, these folks are probably making at least 75k a year. Now, this team is burning up about 2k a day.
So, if these people spend 2 solid weeks at your website, it might cost 20k, and the company would make 15k. It sure seems like a lot of money, but it’s not out of the ballpark.
For those who didn’t have the opportunity to enjoy the “Fill in the graph” question on Bonnie’s second Methods of HCI test, the graph below puts Braden’s double-expert into context. The x-axis is the number of evaluators and the y-axis is the percentage of problems they collectively find:
So given Braden’s team hypothetical team of 3 researchers, double-experts (experts in usability and the interface domain) will find about 15% more problems than usability experts without domain expertise would. Is it worth the extra $15k? I personally wouldn’t pay it, but then, I’m a poor grad student. I need that $15k to pay for Project. And besides, having Jakob Nielsen evaluate a personal website is probably overkill.
Upon reconsidering, it would be a pretty good deal for companies that don’t have their own usability gurus in-house – double specialists dramatically outperform novices. But then, aren’t these the companies that are least likely to realize their need for it?