Screen Space 18: Usability & Usability Testing 101 Part 4—Preparing the Testing
Preparing audio to download.
Sponsored listening. Audio will begin within seconds...
Escucha sin esperasHazte Premium
Preparing audio to download.
Sponsored listening. Audio will begin within seconds...
Escucha sin esperasHazte Premium
Welcome to Screen Space, your podcast about creating usable, accessible, effective, and efficient web, blog, and digital media design for the everyday (and non-expert) designer. This is episode 18 of Screen Space “Usability & Usability Testing 101 Part 4—Preparing the Testing.” In this episode, I discuss the third step of usability testing—preparing the testing. There will be two more parts to this series, where I will discuss conducting the testing and then analyzing and utilizing the results from the testing.
If you have not listened to the previous parts of this series, you may want to go back and listen. In the first part, Screen Space 11: Usability & Usability Testing 101, I discuss usability, provide a definition of usability testing, and outline the steps to conduct a usability test. In Part 2, Screen Space 12: Usability & Usability Testing 101 Part 2—Selecting Users, you can find information on selecting your users for usability testing. In Part 3, Screen Space 17: Usability & Usability Testing 101 Part 3—Deciding what to Test, I discuss the steps to setting objectives and selecting tasks to test. You may also find Screen Space 10 on User-Centered Design helpful.
I am your host, Dr. Jennifer L. Bowie. I conduct research and have taught in areas related to digital media, web, and blog design. Previously I mentioned being an assistant professor at GSU. However, this is no longer the case and I am currently looking for a job in usability, user-centered design, and/or social media. Stay tuned and I’ll provide details at the end of this podcast.
A warm welcome this week to my new listeners from Liverpool, London, and Manchester in the UK. Welcome to Screen Space and design well! Another warm welcome to my loyal listeners. Welcome to another episodes and thanks for listening!
In this episode, I will present the next step in usability testing: preparing for the testing. This includes choosing the order of tasks, creating written test materials, recruiting participants, defining team members’ roles, creating a written test plan, practicing the testing, and preparing the test environment. I will use the same example I used in episodes 11, 12, and 17—testing a photography blog. We’ll imagine we have a photography blog with a decent sized audience. We want to get more users and see how useable the blog is for your current users.
Without further ado, let’s discuss step three.
In the third step we prepare for the testing by choosing task order, creating written test materials, recruiting participants, defining team members’ roles, creating a written test plan, practicing the testing, and preparing the test environment.
Step 1: Choose order of tasks First, choose the order of tasks. You can start easy, go sequential, or be random. I suggest starting with an easy task to warm your users up and give the testing a positive light. If users start with a hard task they could become frustrated and this may impact the rest of the testing. If your tasks follow a particular sequence, it makes sense to go in the sequence. If you have no other determination, you could randomly select the tasks. In fact, if you randomly select the tasks order for each usability test you run, this will had some rigor to your results as task order will not be impacting results.
In episode 17 (part 3), we came up with five tasks. Let’s start easy, but also work with the bit of sequence we have between the task where we ask them to find a photo they may like to buy and purchasing this photo. Here is my recommended task order:
Task 1: Find a particular photo
Task 2: Find a photography tip
Task 3: Find a photo they like
Task 4: Purchase and download that photo
Task 5: Subscribing to the blog
We start with two easy tasks, go to the two sequential tasks, and end with a task that may be harder. I also separated the two tasks where they are finding photos with the photography tip to give them a slight break between similar tasks and to get them to a different part of the site to begin each of the find photo tasks. This way we can see how they find photos from the main page and from the tip page.
Step 2: Create written test materials Next we have to create the written test materials. This will vary based on test, but could include:
Task list for users: This is necessary so the users know what the tasks are. I will link to a sample in the transcript
Written welcome speech/ Intro to be read to user: It is good to make the testing as consistent as possible. So, I suggest writing up a welcome and introduction to the testing that is read to each user. I will link to a sample welcome/intro in the transcript. Feel free to take this and revise it to work for you.
Consent form: You need to have your users’ consent to participate in the testing and to publish any of the information you get from the testing. Testing results, when published, are normally anonymous, which I recommend. This is a good thing to note in the consent forms. I will put a link in the transcript to a sample consent form too.
Observation forms: If you are observing anything in particular you may want to make special observation forms to easily keep track of this data. You can also use the blank forms I created and linked to in the transcript. Plain paper works well too. Use whatever works best for you and your data.
Pre-task & post task questionnaires and interview questions: Often people will want to know demographic and experience level information from the users in the testing. This makes a good quick pre-testing survey. You may also want to conduct a brief interview after the testing. Think about the information you need to understand the results and make surveys or interview questions so you can obtain this data.
Other materials: Think about any other materials you need and make sure they are ready to go.
Step 3: Recruit participants & determine “payment” Of course, you also need to get participants to be the users in your testing. In episode 12, we determined the user profiles and now you have to figure out how to get participants from those user profiles. You could post an invitation on your website, blog, or other digital media. You could use social networking to send out a call for participants. If you have your users’ emails, you could email them. You could go with old media and post a “users wanted” ad in the newspaper. Some companies will use temporary agencies or hiring firms to recruit participants for them. These firms will often do a good job finding participants who fit well into your user profile. Depending on the level of testing you are doing and the users you want to test you, could beg friends and family to do the testing. There are many ways to recruit. Think about what will work best to get the participants you need. For the photography blog, a few blog posts asking users to participate and possibly some posts to the associated Twitter feed may work well.
One thing that may impact recruitment is payment. Often users in usability testing are paid in some form. A local school that conducts usability tests pays students $50-100 for their time. Google recently offered users a choice of $100 Amazon gift certificate, $100 Google Store credit, or $100 donation to your favorite charity for a usability study on the Google Maps product. Some companies try to pay users what their time is worth—so possibly their hourly salary or something similar. Others try to cover the user’s expenses, gas, time and so on. Obviously usability testing can get expensive fast. If you have a smaller site, blog, or digital media you can go with lower cost payments. My students tend to pay each other for testing by providing cookies or cupcakes. If your audience is hungry senior college school students, you could offer beer and pizza. You could provide small gift certificates to something like Amazon or iTunes. If your site, blog, or media has some sort of associated product, you could offer that or a discount in payment. If you are a band, you could give your users a CD and t-shirt for testing, which may also make them bigger fans. For the photography blog, we could give them some of the photos, possibly the photo they choose to purchase in the two related tasks. You can also offer to pay them with your services. If you are a good editor, you could offer to edit something. Think about how your skills can help them.
If you cannot pay your participants much, do not worry. There has been some research that has found that to some degree a “less is more” principle applies to paying participants. A study found that participants who were paid less tended to put more into the study then those who were paid more. The researchers theorized that the lower payment lead to greater investment, as if the participants felt that since they were going to only get a small amount for their time they were going to make it good—and larger amounts had the reverse impact. I’d love to cite this study, but I cannot find source information for this study. But, Festinger et al., in another research study, found that that higher payment amounts lead to increased follow-up rates, which may be important if you will be doing any sort of follow up.
Step 4: Define team member’s roles If you are lucky enough to be testing with other testers, it is a good idea to decide who will be doing what. Here are some traditional roles:
Facilitator/Briefer (necessary): Often the only team member to interact with users. This person will read the welcome/intro, lead the testing, answer user questions, and so on.
Observation recorder/note taker (necessary): This can be combined with the Facilitator/Briefer role. This person will take notes and observe.
Camera operator (optional)
Help desk operator (optional)
Test administer (optional)
It is quite possible to conduct usability testing with a single person. I have done it, and you can too. But it is easier if you have help.
Step 5: Create a written test plan The test plan lays out the full plan for the testing from purpose and objective to the list of tasks. Writing this out will help you see the full picture and make sure everything works together. It also helps organize the testing, which is especially important if more than one person will be involved. I have linked to a sample written test plan in the transcript.
Step 6: Practice It is important and incredibly helpful to conduct walkthroughs of the testing and if possible pilot test. Pilot testing is simply running the full usability test on someone as practice. This is often done in research and usability testing just to make sure everything works as it should. When I have conducted pilot testing, I have learned important things like some tasks were poorly written and unclear, the task order was awkward, and that I needed different supplies than I planned on. You do not have to conduct pilot testing with a real user. It can be a team member if you have a team (ideally one less involved with the test design), a friend, or a loved one. My husband was my pilot tester for my dissertation usability testing.
Step 7: Prepare test environment (day of test) This leaves us with one final step to preparing the testing—preparing the test environment. You want to do this the day of the test or as soon before as possible. This simply means setting up the environment. Turn the computer on and put it on the starting webpage. Set up any cameras and test them to make sure they work. Put the paper tasks out near the computer in order. Put out the consent form with a working pen. Get your clipboard and observation forms ready to go. Do whatever else you need to do to get the testing environment ready for the participants. Once you are setup it is time to welcome your first user participant and begin testing.
Which moves us to the next episode; “Conducting the Test—part 5” in the Usability Testing series. Come back next week to learn how to conduct the usability testing. To prepare for testing, remember you need to:
Choose order of tasks
Create written test materials
Recruit participants & determine “payment”
Define team member’s roles
Create a written test plan
Prepare test environment (day of test)
That concludes Screen Space episode 18: “Usability & Usability Testing 101 Part 4—Preparing the Testing.” Do join us next week for the next installment of this series: Screen Space 19: Usability & Usability Testing 101 Part 5—Conducting the Testing.” In two weeks, I will conclude the series with the fifth part: “Analyzing and Utilizing the Results.”
As I mentioned in the intro to the podcast, I am looking for a job. As my loyal listeners may be able to guess, I am interested in a position in usability, user-centered design, and/or social media, or another academic position teaching these areas. My preference is in the Atlanta area or telecommuting, though I may consider locations somewhat nearby. If you are interested in my skills or know someone who is please contact me at firstname.lastname@example.org and check out my portfolio at www. screenspace.org/port.
If you have questions, comments, or thoughts on what you want me to cover please send me an email at email@example.com or check out the Screen Space blog—www.screenspace.org. You can also follow Screen_Space on Twitter for hints, tips, advice, news, and information on designing websites, blogs, and other digital media texts. Also, check out the blog for a transcript of this podcast complete with links and resources.
Have fun and design well!
Screen Space is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License. So, feel to send a copy to the friend who kindly acted as your guinea pig and helped you pilot test, but don’t change the podcast, do give me and Screen Space credit, and don’t make any money off of it.
Screen Space’s opening music today is “African Dance” by Apa Ya off of Headroom Project and the closing music is “Survival” by Beth Quist off of “Shall We Dance”. Both these selections are available from Magnatune.
Episode 18 Links and References: Past Screen Spaces podcasts you may want to refer to:
Screen Space 10: User-Centered Design 101 (Why user-friendly is not enough) https://screenspace.org/?p=115
Screen Space 11: Usability & Usability Testing 101 https://screenspace.org/?p=147
Screen Space 12: Usability & Usability Testing 101 Part 2—Selecting Users
Screen Space 17: Usability & Usability Testing 101 Part 3—Deciding what to Test
Festinger, David S, et al. “Do research payments precipitate drug use or coerce participation?” Drug and Alcohol Dependence, 78.3, 1 June 2005, 275-281.
Usability Testing Resources: Written welcome speech/intro to be read to user: https://screenspace.org/wp-content/uploads/welcome.doc
Consent form: https://screenspace.org/wp-content/uploads/consent_form.doc
Observation form: https://screenspace.org/wp-content/uploads/ob_form.doc
Sample task for user to read: https://screenspace.org/wp-content/uploads/ExampleTask.doc
Written test plan: https://screenspace.org/wp-content/uploads/SampleTest.doc
Other links: Magnatune: https://www.magnatune.com/
Subscribe! Subscribe to the blog
Subscribe to the podcast