Task measurement webinar - my summary

A couple of nights ago I was up at 2am to
listen to Gerry McGovern (www.customercarewords.com) and Rolf Molich (www.dialogdesign.dk)
talk about how they quantify task measurement.

I've listened to a couple of Gerry's web
casts in the past and I think this one really made a very good compromise
between providing useful information, and promoting the products Gerry sells
via Customer Care Words.

The following is my take on what Gerry and
Rolf were trying to get across.

Two really useful little things to begin
with - a user click takes 4 to 5 seconds and to gain executive traction show
user experience before and after.

Gerry spoke about failure of intranets
based on tools and content.

Circa 2000 IT managed an intranet full of
tools and the post 2000 intranet stuffed full of content managed by the
communications team "like giving a pub to an alcoholic" - Gerry McGovern

Gerry sees intranets today being TASK
focused - the task solving the content / tool problem by giving it a
business focused context.

Task Management

1.            First phase discover what the
top tasks are

2.            Second phase define the
efficiency of the top tasks

3.            Third phase improve the
efficiency of the top tasks by iterative testing and change

Gerry outlined how he works by identifying
an expected click path a user will take to complete a task, and using this
to test against.

User testing will then show up the other
possible click paths, quantify these with 15 users. The number who follow
each click path provides DATA which is much more powerful than opinion.

DATA rather than opinion and user testing
puts the user at the center of change and allows for evolution as users
change.

At this point Rolf started his presentation
on the method they use for task measurement.

               Re writing a well known
quote Rolf comes up with "If you can measure it you can manage it"

Thus - "Time and attention are the currency
your users spend on your website/intranet"

The methodology Rolf follows :

1. Identify top 10 tasks, measuring 5
   costs the same as 10. Often there are 3 to 5 really big tasks each with
   two to five sub tasks


2. Develop measures


3. Assign responsibility to top users
   or task managers for improving efficiency (make their bonuses dependant
   on it)


4. Report results


5. 
   Improve performance

Task measurement is a management tool that
provides a framework for systematic treatment of task failures and
disasters, two areas that are often overlooked using ad-hoc processes

1 Top Tasks

Determine top tasks
by doing

• a carewords
  study


• most used pages


• looking at
  standard tasks across intranets e.g. find people

2 Recruit
participants

15 to 20 users who
are representative of your site / intranet , or the part of your site that
you're working on

More than this
number tends to be un-economic, less than this number doesn't give a sample
size big enough to represent the user base as a whole

3 Do measurement
sessions

Emphasis to the
users that you're measuring the site/intranet rather than them.

Use a stop watch,
and time each user doing all the tasks as quickly as possible.

Their comments are
useful, but they slow the user down so don't ask for them to tell you what
they're thinking, watch what they're doing. Rolf commented that he finds out
the same amount about usability with or without the comments.

30 to 90 mins
should be enough for to observe 10 to 20 tasks

Debrief with users
at end of measurement sessions.

4 Compute and
report the results

Collate the
following :

Success - time it
took to do task successfully

Failure - user
gives up / user can't do it within 6 mins / technical problems prevent
completion

Disaster - task
completed but the answer is wrong and believed by the user

Calculate the
following :

• Completion rate
  (effectiveness)


• Completion rate
  time (efficiency or the median time for task)


• Disaster rate


• Optimal time
  (Gerry's idea) - this is the projected time for task to be completed with
  improved user interface and realistic response times


• Task
  Performance Indicator (TPI)

TPI = success
factor X (optimal time for task / completion rate time)

e.g.

Optimal time
(estimated) = 40sec

Median time (test
result) = 100sec

Optimal / median =
40%

Success factor =
60% that is 9 out of 15 staff completed the task successfully

TPI = 60% X 40% =
24%

Report the
following :

• Success


• Failure


• Disaster


• TPI
  

Note - some senior
executives only want to see the TPI value

5 Improve the
task performance

Using general
usability and specific usability make the task better and faster.

Exactly what you do
here is based on best practices, your own experiences and your understanding
of the organisation.

6 Measure again

For the same tasks
with different users go through the process again.

Typical results

Disaster 7 %

Failure 32 %

Success 61 %

Main reasons for
failure

1. search


2. search


3. search


4. navigation


5. freshness - the
   data is outdated

Productivity
gains

Example

Find a person 900
times / year / person | typically takes 40sec | optimal time 20sec

Potential savings
for an organisation of 36,000 people 180,000hrs per year to the organisation

To quote Gerry
"Productivity gains - speak the language of the manager"

To quote Gerry
again "the modern economy is all about time!"

Summary

Total Productivity
Index (TPI) relies on the same task being done many many times e.g. a few
staff doing the same thing a lot, or lots of staff doing the same thing a
few times.

Personally I really
like the idea of beginning with a hypothesis and then moving on to see how
true that is in the real world. Quantification of that means that
comparisons can be made allowing the intranet to start to speak the same
language as the rest of any organisation, time being money and all.

The other think I
liked about the presentation was that both Gerry and Rolf encouraged those
attending the session to go and try it for themselves, that in fact it
wasn't rocket science, but eminently doable with a stop watch, some users
and a calculator. Being a kiwi this really appeals to my number eight wire
mentality of getting things done with the tools at hand.

What did it inspire
me to do - well I've identified the top most visited parts of my intranet,
so now I'm off to write up my hypothesis and do some user testing.

DorjeM