Help Test D7UX – Crowd-sourcing Usability Testing

If you took a look at our outline for the project process the other day, you’ll know that we’re taking an iterative and user centered approach to this project. That means that at regular intervals through out the project, as well as doing all this online stuff, we’re also going to sit down with real people in ‘real life’ and let them use our design work while we observe them and make notes about what is working well and what requires improvement. This is sometimes known as Usability Testing.

Now, Mark & I are going to be doing a bunch of this ourselves, and our friends in the Drupal Usability Group are also going to be helping out as much as they can (thank you!), and Jeff at Acquia will also be helping out – but (as we did with the d.o redesign), we’d like to invite YOU to help out with some usability testing as well!

Why on earth would you want to do this?

As I see it, there are three main reasons why you’d get involved in something like this:

  1. you’d like to get some (or some more!) experience in usability testing. This is an opportunity get some experience under your belt, or find out what doing usability testing is like. You may have students or interns who need to get some experience, here’s a great project to keep them busy AND use their time to contribute to a great project.
  2. you want to see for yourself how other people interact with the design. One of the main reasons that we do usability testing is that it is very easy to design for ourselves and often incredibly difficult to design for other people. People come to Drupal with all kinds of backgrounds and understandings, and very often what is clear as day for us is impossible to understand for them. (See the video we posted of Mark & I installing Drupal for some evidence to this point).
  3. you love Drupal and this project and you desperately want to see it succeed: one of the key ways to reduce the risk of failure of this project is to get it out in front of people and see what they make of it. Not only does this ensure that we’re not designing and releasing something that is broken and unusable, it makes us feel confident that the decisions we are making are the right ones and that they will have a significant and positive impact on the user experience of Drupal.

As we move forward with this redesign and start to make some concrete decisions, we are going to have some tough discussions about ‘what users do’ – the BEST way to have these discussions is with *real* experience of our future end users, and not just sweeping statements about users in general. Help us build up that body of knowledge and inform yourself in the process.

How will it work?

As you can imagine, we don’t do this kind of thing every day. We tried this on the d.o redesign project and whilst lots of people voiced interest, the only person who actually did any testing was me! However, we still think this is a good idea and we suspect that, with a little more structure and a fixed schedule, it might work. So, here’s the current plan:

  • one week before Test Time we will release a description of the audience type(s) that will be appropriate for this round of testing. You should then recruit as many people who fit that description as you think you will have the time and/or inclination to interview/test. You should allow around an hour for each interview, but don’t schedule more than 4 or 5 in a day (trust me, your brain will melt and you will make a LOT of data to analyse)
  • a few days before Test Time we will release:
    • the Prototype that you will be using to test (starting off with PDF/Paper prototypes and moving onto web-based interactive prototypes as the project continues).
    • an Interview Guide that you will base your interview on, listing all the key tasks and issues we want you to cover in the interview
  • During Test Time (or as close to it as you can) you conduct your interviews. If you can (please!) record your interviews on video and post either the entire video or some highlights from it to the YouTube Group (or elsewhere if you prefer). We will also provide a place for you to log your test findings (TBC, we’re looking at a few options here, ping me if you have suggestions!)
  • We will then include your findings in our analysis of the design work to date and use it to inform our design decisions going forward.

The Testing Schedule:

We have penciled in some dates for ‘Testing Time’ throughout the project… these are subject to change, but if you are thinking of participating, perhaps see if one or more of these dates works for you:

  • 6-8 April
  • 15-16 April
  • 27-29 April
  • 18-20 May
  • 1-3 June
  • 29 June – 1 July

Need more help?

Over the coming weeks I’ll post some more information that will help provide support for those who aren’t seasoned researchers. Things I’m thinking of posting include: tips for a good interview, tips for recruiting, suggested software/hardware for recording/editing sessions… can you think of anything else that would be helpful?

Thank you!

I’m really excited about this exercise and I hope you are too. Please leave a note below if you’re interested in participating or if you have any questions, need more information, have any feedback on the proposed process. If you know of people in schools or companies who might be interested in participating, please send this on to them!

I really look forward to working with you on this!


21 thoughts on “Help Test D7UX – Crowd-sourcing Usability Testing”

  1. Will definitely get involved with this! Great usability testing experience alongside helping out a great project, which although good from a development perspective was lacking Front-End direction.

    I’ll make sure to check out both of your Twitter accounts for updates.

  2. I love this idea and would like to help out. I have a number of people at work with varying levels of Drupal experience that I could use as testers, I might just have to bribe them a little to participate!

  3. Sounds like a good idea and an easy thing to crowdsource. I’m in.

    Besides, if @garrettc is getting involved, I dare say he’ll rope me in too. A volunteer is worth ten pressed men.

  4. Yes, I can to do some tests too. Paper + video no problem and later-on I can use by Silverback too. However the test will probably be conducted in Dutch, so we need to find a way to get you the right feedback from the test.

  5. My though was, YES I want to help – but spending two days interviewing people – I’m not sure if I want/can invest that much time.

    So is there a way to do just one,two or three interviews during the next few weeks? Spending an evening once in a while would be OK for me, and I guess many others.

    Could you elaborate on what the minimal investment would be?


  6. I recently came across your blog and have been reading along. I thought I would leave my first comment. I don’t know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.


  7. Is the idea that some people would participate in one or several Testing Times – not all of them? Taking part to some would certainly be possible and interesting.

  8. Leisa, I’m planning on doing some testing in the future rounds of testing. I only just got back from holiday and won’t be organised for this round (6-8 April). I’ve already marked the others on my calendar though.

    Please can you blog about software & hardware, and recruiting. Thanks!


  9. Hey, I am currently studying MA interactive media, and usability testing is a big part of the degree, so getting some experieince would be great. 🙂
    I am available for the may, june, and july dates.



  10. This sounds like a very interesting project. Please count me in. I am available for any of the May, June & July dates.

    Let me know

    Thank you,

Comments are closed.