Finding Usability Testers: Tips from an Army Recruiter

After spending 22 years in the U.S. Army, including 3 years as a recruiter, Julie Jackson realized that not only was she qualified to work in usability, but had a knack for it—especially because of her ability to strike up a conversation with nearly anyone, anywhere. Julie shares how her training in the Army has helped in her approach to usability testing, and gives a peek inside how usability testing works for USAJOBS.gov at the Office of Personnel Management (OPM).

Tell me about your career in the Army.

I spent 22 years in the Army, all of which were in the enlisted ranks. I started my career as a dental assistant, and after 10 years, was called to recruiting. I spent 3 years as a recruiter in Muskogee, Oklahoma and then attended the Career Counselor course. I spent the rest of my Army career as a Career Counselor where I negotiated reenlistment contracts and assisted Soldiers in their transition from military service to civilian life.

How did you find usability?

I can’t say that I found usability because honestly, I had never heard of it before I came to OPM. I was looking for a federal career after I retired from the Army. I was working as a contractor for another agency. One of my co-workers had just been hired under the Pathways program, and I asked her for some information. That same day, I started looking for jobs in the Pathways program and one day found this interesting job announcement with USAJOBS. I thought, I have done all of that (the duties) and it looks like I am qualified, so I applied.

When I got my first interview I was really intrigued. I was so excited when I got my second interview and even more excited when I got the call that I was selected for the position. I had an amazing mentor who explained usability and customer experience, and I immediately attended the human centered design class in the Lab at OPM. Once I understood what usability and human-centered design was, I realized I had been using those concepts for a long time and just didn’t know that it had a name.

I started working for USAJOBS in October of 2014, so, I have been here almost 3 years now and have enjoyed coming to work every day. I get to talk to people every day about a product that I believe in and I know that the changes we are making is making a difference in the way people search and apply for federal jobs.

Tell me about a typical usability test for USAJOBS.

We try to do a couple of hallway tests using a wireframe to identify major changes to functionality and/or design, and then we like to do one laboratory test using a fully functional prototype prior to sending anything to our developers. Depending on the audience, we will either do an in-person test or conduct the test via WebEx. We have conducted some card sort exercises when we are working on a UI issue, but try not to do unmoderated tests. Because Human Centered Design is the foundation of our research, we feel it is important to be able to see the participant’s facial expressions and/or hear the inflection in their voices. This will alert us of some user frustration even if they don’t want to tell us something is wrong.

We usually have between 8 and 10 participants for a hallway test. These tests are short and we usually conduct them during lunch time so we don’t take participants away from their assigned duties. For a laboratory test, we will test 7 or 8 people. These tests are usually conducted the entire day and they last anywhere from 45 minutes to an hour.

How do you recruit potential testers?

During our initial research, we compiled a list of volunteers; however, that list is a little stale since we have been using it for about 2 years. I talk to anyone that will listen and ask everyone to participate in testing. One day I walked out the front of the building and talked to people who were waiting for their commuter bus to arrive. This was a short notice test and I knew that if they were waiting for a bus outside my building that they worked nearby. I have also talked to people in line at the food trucks. 😊

Any time we have a meeting with a group of people who work outside of OPM, I always take the opportunity to pitch usability testing and have a sheet available for people to give me their contact information.

Here are some examples of how we’ve recruited for specialized needs and interests:

Students: I have reached out to several colleges in the area and asked if I could come to their campus to conduct usability tests. I usually start at the office that helps students find jobs after graduation and make sure I have a point of contact before I leave the campus.

Vets: I reached out to the transition office at Fort Belvoir and have been allowed to come to their building to recruit and conduct usability tests. I also sit in on some of the transition classes so that I can ask for testing volunteers prior to the start of the class.

Federal employees: I have identified people at various agencies that can give me permission to conduct usability tests at their location.

HR Specialists: When we have a product that will be used by HR Specialists, I go to USAJOBS and look at job announcements. At the bottom of each job announcement there is contact information, so, I call that person and ask them to participate in testing. We also ask for volunteers during our Agency User Group monthly meetings (I get both HR Specialists and federal employees this way).

How do you organize your testing leads?

For laboratory tests, I use a spreadsheet. This way I am able to note the last time someone participated and what they tested. Using a spreadsheet allows me to track participants and ensure that the same people aren’t testing all of the time.

How has your career as an army recruiter helped you approach and excel at the task of recruiting usability testers?

As an Army recruiter, I had to talk to anybody that would listen so that I could accomplish my monthly mission requirements. I always tell people that “I was called everything but nice” every single day. I had to stand strong in my loyalty to my service and not let others stand in my way. I use the same mindset when I recruit for a usability test.

I don’t take any criticism personally. If someone complains about the site I simply tell them, “That’s great. By participating in this usability test you will be able to help us make the site better.” I also know, from my recruiting days, that you have to reach out to more people than you have slots. For example, if I need 8 people, I will send out at least 16 emails—sometimes more. I understand that people will say they want to participate but when they get my email, they may be busy or they may have changed their mind.

In the Army, things didn’t always go as planned. I use those experiences to ensure that no matter what happens, the test must go on. We may have technical difficulties, the space may not be set up when we get there, or someone from the usability team may be sick that day. Everyone on the usability team has been trained to fill all of the roles needed to conduct a test so if someone is out, someone else steps up and fills that role (this is just good Army training, not necessarily recruiter training).

In closing, what advice would you give to another usability expert in government who is struggling to find testers?

Talk to everyone. Never let an opportunity to recruit pass you by. In the Army, I was always recruiting; when I was at a restaurant I would talk to the waiter/waitress, when I was at the ER one time with a broken ankle, I talked to guy taking my vital signs, and when I was making a medical appointment the other day I talked to people making my appointment. I don’t expect everyone to do this, but, if you can you will find people who want to help.

Always ask for referrals. This is a great way to find people you may otherwise never encounter. Ask open an open ended question like “who do you know that would be interested in helping us out?”

Always have something to write with and on. You never know when someone is going to say “I want to help” or “This guy I work with would probably be interested”

When sending out your emails requesting volunteers, talk to them as if you were on the street, doesn’t use government speak. Let them know their participation will make a difference. “Help us redesign USAJOBS” “We value your feedback” “Your feedback will directly affect our design decisions” or “We want to make sure that you will be able to use the new feature”.

Build relationships with people who have access to large groups of people. Meet with them face-to-face to explain the importance of usability testing. It is hard to explain what this is to someone who has never heard of usability testing before.

Don’t take anything personal. You will always have your critics, but you will always have those who want to help—find those people.