Guerrilla User Testing July 27th, 2011

Project overview: Having worked for the client a number of times my relationship grow from an initial piece of work to run UAT (User testing) of one of the clients many websites. The site having been live for six months was at a suitable point at which to run user testing to direct the next phase of development identifying where best to focus efforts.

Key deliverables

• Several key usability issues uncovered in under 10 days
• Actionable recommendations presented to stakeholders
• Findings documented with show reel, report and presentation slides
• Recruitment costs: 9 coffees, 2 teas and 4 cakes, 1 caramel wafer (high cost due to Starbucks location)

Key areas of interest

User testing, Usability review, Guerrilla User Testing

Testing approach and design

The primary aim for the testing was to identify usability problems whilst at the same time answer a series of opinion based questions related to brand and content. In consultation with the team responsible for site delivery and ongoing maintenance my employer (Digirati) rapidly designed a test script that would capture the required information, an approach was designed combining two practices.

The first component was a series of observational tasks to gain insight into how the participant went about discovering content available on the site. Screen capture software recorded the participant’s interactions with the live website whilst a webcam captured the all import sound bites and facial expressions that further illustrate personal reaction to particular scenarios.

Opinion was captured through a series of questions each assigned with a unipolar 5 point scale designed to prompt the respondent to think of the presence or absence of qualities and attributes. These questions allowed subsequent analysis to ascertain the participant’s opinion regarding branding, look and feel and quality of content.

Guerrilla user testing - for the price of a coffee

To recruit the audience describe as the ‘entire population’ could have been an expensive time consuming task, however. Using the Guerrilla approach for user testing resulted in the collection of a large sample of data from a stream of participants from the general population within a short time frame. Quickly it became apparent that all participants struggled with a consistent set of features and a particular set of labels and terms used to categories some content. Some of the observations were expected, other as always proved surprising. This confirms that prior knowledge will successfully get you only so far. Prior knowledge only really applies in context and context is vital in successfully communicating and commercially transacting with your audience.

Actionable deliverables

• A multi format show reel highlighting key findings form the Guerrilla testing sessions was accompanied by a detailed report both of which were presented to the team at their monthly project board meeting. As a result the following actionable items resulted.
• Detailed user stories that enriched the personas used to inform design decisions.
• Observed user behavior based recommendations to improve the presentation of information resulting from the use of the free text search field.
• Confirmation that the content published was considered high quality, informative and most importantly trusted by users.
• Highlighted that some terms used across the site were not best suited to promote certain classifications of content.
• Discovered that some key navigational elements were either missed or disregarded by participants because of their similarity to online advertisements.