Our UX Principles: 1. Make the most frequent tasks easy and less frequent tasks achievable. 2. Design for the 80% 3. Privilege the Content Creator 4. Make the default settings smart

Help Test D7UX – Crowd-sourcing Usability Testing

Posted: March 31st, 2009 | Author: | Filed under: Usability Testing | Tags: , , | 21 Comments »

If you took a look at our outline for the project process the other day, you’ll know that we’re taking an iterative and user centered approach to this project. That means that at regular intervals through out the project, as well as doing all this online stuff, we’re also going to sit down with real people in ‘real life’ and let them use our design work while we observe them and make notes about what is working well and what requires improvement. This is sometimes known as Usability Testing.

Now, Mark & I are going to be doing a bunch of this ourselves, and our friends in the Drupal Usability Group are also going to be helping out as much as they can (thank you!), and Jeff at Acquia will also be helping out – but (as we did with the d.o redesign), we’d like to invite YOU to help out with some usability testing as well!

Why on earth would you want to do this?

As I see it, there are three main reasons why you’d get involved in something like this:

  1. you’d like to get some (or some more!) experience in usability testing. This is an opportunity get some experience under your belt, or find out what doing usability testing is like. You may have students or interns who need to get some experience, here’s a great project to keep them busy AND use their time to contribute to a great project.
  2. you want to see for yourself how other people interact with the design. One of the main reasons that we do usability testing is that it is very easy to design for ourselves and often incredibly difficult to design for other people. People come to Drupal with all kinds of backgrounds and understandings, and very often what is clear as day for us is impossible to understand for them. (See the video we posted of Mark & I installing Drupal for some evidence to this point).
  3. you love Drupal and this project and you desperately want to see it succeed: one of the key ways to reduce the risk of failure of this project is to get it out in front of people and see what they make of it. Not only does this ensure that we’re not designing and releasing something that is broken and unusable, it makes us feel confident that the decisions we are making are the right ones and that they will have a significant and positive impact on the user experience of Drupal.

As we move forward with this redesign and start to make some concrete decisions, we are going to have some tough discussions about ‘what users do’ – the BEST way to have these discussions is with *real* experience of our future end users, and not just sweeping statements about users in general. Help us build up that body of knowledge and inform yourself in the process.

How will it work?

As you can imagine, we don’t do this kind of thing every day. We tried this on the d.o redesign project and whilst lots of people voiced interest, the only person who actually did any testing was me! However, we still think this is a good idea and we suspect that, with a little more structure and a fixed schedule, it might work. So, here’s the current plan:

  • one week before Test Time we will release a description of the audience type(s) that will be appropriate for this round of testing. You should then recruit as many people who fit that description as you think you will have the time and/or inclination to interview/test. You should allow around an hour for each interview, but don’t schedule more than 4 or 5 in a day (trust me, your brain will melt and you will make a LOT of data to analyse)
  • a few days before Test Time we will release:
    • the Prototype that you will be using to test (starting off with PDF/Paper prototypes and moving onto web-based interactive prototypes as the project continues).
    • an Interview Guide that you will base your interview on, listing all the key tasks and issues we want you to cover in the interview
  • During Test Time (or as close to it as you can) you conduct your interviews. If you can (please!) record your interviews on video and post either the entire video or some highlights from it to the YouTube Group (or elsewhere if you prefer). We will also provide a place for you to log your test findings (TBC, we’re looking at a few options here, ping me if you have suggestions!)
  • We will then include your findings in our analysis of the design work to date and use it to inform our design decisions going forward.

The Testing Schedule:

We have penciled in some dates for ‘Testing Time’ throughout the project… these are subject to change, but if you are thinking of participating, perhaps see if one or more of these dates works for you:

  • 6-8 April
  • 15-16 April
  • 27-29 April
  • 18-20 May
  • 1-3 June
  • 29 June – 1 July

Need more help?

Over the coming weeks I’ll post some more information that will help provide support for those who aren’t seasoned researchers. Things I’m thinking of posting include: tips for a good interview, tips for recruiting, suggested software/hardware for recording/editing sessions… can you think of anything else that would be helpful?

Thank you!

I’m really excited about this exercise and I hope you are too. Please leave a note below if you’re interested in participating or if you have any questions, need more information, have any feedback on the proposed process. If you know of people in schools or companies who might be interested in participating, please send this on to them!

I really look forward to working with you on this!

Leisa


21 Comments on “Help Test D7UX – Crowd-sourcing Usability Testing”

  1. 1 garrettc said at 11:17 am on March 31st, 2009:

    Count me in.

    For anyone else thinking of diving in http://silverbackapp.com/ is a great option for usability testing if you’re on a Mac.

  2. 2 tdskate said at 11:27 am on March 31st, 2009:

    I was about to say the same thing, garrettc. Silverback ftw.

  3. 3 Kushal Pisavadia said at 11:49 am on March 31st, 2009:

    Will definitely get involved with this! Great usability testing experience alongside helping out a great project, which although good from a development perspective was lacking Front-End direction.

    I’ll make sure to check out both of your Twitter accounts for updates.

  4. 4 steve said at 1:06 pm on March 31st, 2009:

    I love this idea and would like to help out. I have a number of people at work with varying levels of Drupal experience that I could use as testers, I might just have to bribe them a little to participate!

  5. 5 J-P said at 1:09 pm on March 31st, 2009:

    Sounds like a good idea and an easy thing to crowdsource. I’m in.

    Besides, if @garrettc is getting involved, I dare say he’ll rope me in too. A volunteer is worth ten pressed men.

  6. 6 Sutharsan said at 1:12 pm on March 31st, 2009:

    Yes, I can to do some tests too. Paper + video no problem and later-on I can use by Silverback too. However the test will probably be conducted in Dutch, so we need to find a way to get you the right feedback from the test.

  7. 7 Bojhan Somers said at 3:09 pm on March 31st, 2009:

    So is this on your prototypes?

  8. 8 Leisa Reichelt said at 3:19 pm on March 31st, 2009:

    thanks everyone!

    @bojhan – yes, we’ll provide prototypes (paper at first, then a web based prototype) to test.

  9. 9 Jeff Noyes said at 9:12 pm on March 31st, 2009:

    related, Adobe gave the Drupal community 500 free Connect Pro licenses. You can read more here: http://acquia.com/blog/crowd-sourcing-usability-testing-free-adobe-connect-pro-licenses

  10. 10 Jan said at 8:39 am on April 1st, 2009:

    My though was, YES I want to help – but spending two days interviewing people – I’m not sure if I want/can invest that much time.

    So is there a way to do just one,two or three interviews during the next few weeks? Spending an evening once in a while would be OK for me, and I guess many others.

    Could you elaborate on what the minimal investment would be?

    Jan

  11. 11 Drupal 7 User Experience Project » Blog Archive » Current Activities - What you can do NOW! said at 9:59 pm on April 1st, 2009:

    [...] Crowd-sourced Usability Testing: help us get the D7UX tested all over the world and get some usability testing experience under your belt – the first testing ’session’ is coming up on 6-8 April [...]

  12. 12 Ruth said at 4:31 am on April 3rd, 2009:

    I recently came across your blog and have been reading along. I thought I would leave my first comment. I don’t know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.

    Ruth

    http://fendisite.com

  13. 13 k said at 6:44 am on April 3rd, 2009:

    Is the idea that some people would participate in one or several Testing Times – not all of them? Taking part to some would certainly be possible and interesting.

  14. 14 michelle said at 9:25 am on April 4th, 2009:

    would be happy to help! i’m available either the 27-29 of april or either dates in june.

  15. 15 Drupal 7 User Experience Project » Blog Archive » Crowdsourced Usability Testing Rnd 1 - Investigating Information Architecture said at 12:35 pm on April 6th, 2009:

    [...] today is supposed to be our first day of Crowd Sourced Usability Testing – you might have noticed that we’re running a little behind on this, so to make up for the [...]

  16. 16 Saylor said at 3:02 pm on April 6th, 2009:

    Would love to help out. Count me on board.

  17. 17 Emmrat said at 5:59 pm on April 6th, 2009:

    I am available for the 29th June -1st slot.
    I would really like to be involved, thanks!

  18. 18 Bevan said at 10:28 am on April 7th, 2009:

    Leisa, I’m planning on doing some testing in the future rounds of testing. I only just got back from holiday and won’t be organised for this round (6-8 April). I’ve already marked the others on my calendar though.

    Please can you blog about software & hardware, and recruiting. Thanks!

    Bevan/

  19. 19 Karima said at 2:00 pm on April 8th, 2009:

    Hey, I am currently studying MA interactive media, and usability testing is a big part of the degree, so getting some experieince would be great. :)
    I am available for the may, june, and july dates.

    Thanks,

    Karima

  20. 20 Amit said at 6:22 am on April 29th, 2009:

    This sounds like a very interesting project. Please count me in. I am available for any of the May, June & July dates.

    Let me know

    Thank you,
    Amit

  21. 21 Asaph said at 7:22 pm on June 1st, 2009:

    From what I understand there is still time to say “I’m in!” so – I’m in!

    Keep me updated.

Royalty Free Music | garcinia cambogia where to buy in australia