The second edition of Erika Hall’s Just Enough Research dropped October 2019; although this excellent volume was previously unknown to me I am taking the opportunity now to consume, embody, and evangelize Hall’s approach to user research. Or, as Hall might put it, I’m a willing convert to the gospel of “Enoughening”. Hall is a seasoned design consultant and co-founder of Mule Design Studio but her commercial approach is tempered by a no-nonsense attitude that makes her solutions and suggestions palatable to a small UX team such as my own at Indiana University Bloomington Libraries.
Rather than conduct a formulaic book review of Just Enough Research, I want to highlight some specific things Hall tells the reader not to do in their UX research. This list of five “don’ts” summarize Hall’s tone, style, and approach. It will also highlight the thesis of the second edition’s brand new chapter on surveys. Some of these might be familiar to you and it’s my hope that the small sample of insights I share will convince you to spend more time with Just Enough Research.
#1 Don’t seek “validation”.
This seems like a quibbly point to make, but I have a feeling that one day bringing this up in a meeting is going to make me look wicked competent. “Validation” in research lingo means something specific; it refers to proving a hypothesis right or wrong. Hall posits that using phrases like “validate the design”: “sets the expectation that your goal is be proven right, not to learn.” (30) This small shift in language makes sure emphasis is put on all data; not just data that proves any hypothesis you might have about the product you’re building. Or, as Hall says, “Aim to be proven wrong. And you’ll be far more right in the long run.” (31)
#2 No one needs a UX lab.
“There is no reason to test in anything called a “usability lab” unless there’s a danger your experiment will escape and start wreaking havoc. A usability lab gives you the illusion of control.” (53) This is a fitting rebuttal to the articles I’ve seen published recently about academic libraries allocating space for a UX lab. Especially given the variety of free and cheap technology to conduct remote testing, is a UX room of one’s own superfluous? Hall makes a good case that the answer is yes.
#3 Don’t call things a Likert Scale that aren’t a Likert Scale.
A TRUE Likert scale MUST have:
- A five or seven point range,
- A range of agreement and disagreement (but Hall has a nifty chart that shows other qualities that a Likert can measure),
- A neutral option in the middle (which some researcher leave out: Hall says leave it in),
- Results which are evaluated for statistical relevance. Remember, it is a quantitative method.
- A mean is meaningless; and ranking the categories probably is too. Because Likert data is ordinal and discrete with a limited range, use a bar chart and calculate the mode for better results.
If you think you’re using a Likert scale but it doesn’t fulfill all of those characteristics, what you have is an ordered response scale. Hall is taking me back to my grad school’s introductory research methods class and honestly, I needed the refresher.
#4 Don’t test everyone (craft good screener questions).
Admittedly, I’m bad at this one. My focus is on quick and dirty testing where I take who I can get and use lightweight methods. I don’t want to waste time by starting my usability testing recruits with a survey. Hall has convinced me that screeners lead to better data and it came from this sentence: “You could show them a fully functional, whiz-bang prototype and be met with stares and unhelpful critiques”. (47) I have seen many (MANY) such blank stares from participants who I assume care about the library catalog because they happen to be in the library building. This is a flawed assumption that gives me crappy data. Hall also gives tips for building a screener that gets you the participants you need.
#5 Maybe don’t do a survey at all.
“What’s the worst that could happen?” Hall asks the reader, who she assumes is being pressured, once again, to do a survey by their administrator. Brexit, that’s what. (136) Surveys are deceptively easy, require significant qualitative and/or quantitative analysis, need more responses than you think, and are often used to measure things like “satisfaction” poorly. Read this chapter and you’ll a) run fewer surveys, and b) write much, much better surveys.
A key themes I see in Just Enough Research, both this edition and the first, is that those of us who do UX work often times see our methods co-opted to serve purposes completely unrelated to user needs or good design. Hall presents and explains foundational concepts in the UX field in a way that will help me explain and defend how and when my methods are useful, and when they aren’t.