The Metrics of Usability
If a Recommendation Falls in the Forest, Does
It Make a Sound? March 12, 2005
Synopsis: Is usability productive
work? How productive? Can we prove it to management to justify
our existence? Or is the issue not one of work but results?
The subject repeatedly arises of
how to quantify the efforts of usability experts. Should we measure
the number of tasks that we perform... reviews, tests, recommendations,
designs, mockups, reports... and then pat ourselves on the
back proudly declaring ourselves the most valuable employees
in the company? Or is the real issue results... starting at
the end: increased sales, and working backward from there?
Answer: the latter, and anything short is subterfuge or malicious
compliance, pandering to corporate silliness.
conviction that attempting to reduce usability to numbers is
a telltale sign of a business that is simply
dying but no one knows yet, I nonetheless
accept that many businesses are incapable of tying
decisions, so here are some ideas on genuine metrics:
- Change (inversely correlated) in all costs and logistics
related to training, documentation, and techsupport:
- - duration of calls
- - quantity of calls
- - length, quantity, cost of training classes, internal
- - pages printed, in development and techsupport as
well as for end users
- - copy paper purchased
- - printing costs
- - salary of techsupport
- - years experience required for techsupport
- Change in customer referral rate.
- Change in sales (or web "conversion" rate from
prospect/free to paying customer).
These are all business (outwardly focused) results. Items such
as "how many tests were conducted" are inwardly focused,
and I would hope that as a relatively new field, we are not
already going down that path. I understand the attraction to
such objective measures, but there is a fundamental conflict:
the easier something is to measure and directly correlate,
the less meaningful it is. Sales, the ultimate metric, is easy
to measure but difficult to apportion to usability. Customer
satisfaction is easier to correlate to usability but most folks
in the business hardly recognize a usability showstopper when
it's right between their fingertips and their eyes. How can
we reasonably expect customers to make the correlation?
Usability is about results. Educate those who imply less or
ask you to perpetuate the notion by measuring something inwardly