Dealing with the Quant Obsession as a UX Researcher
Welcome to an infrequent series of writings on UX Research, Behavioral Economics, Human Behavior, Decision Making, Movies and Books.
The following is an illustration of the problem I have heard people (UX researchers) communicate to me over the last couple of weeks.
Consider this, you have run a research exercise trying to find out why people are uninstalling an app after logging in. After having done extensive interviews, you understand the plethora of reasons behind this action and wish to convey the same to the product team for them to take actionable insights. But… and this is a big one, (not the kind which has Sir Mix-A-Lot’s approval), you are met with stiff resistance. Questions are raised for which you have just given answers to, answers you give do not receive the a-ha! response that you expect. The most oft repeated question is, “How many people said this?”, “You want us to do/change this because X number of people said they didn’t understand? But the numbers I have with me doesn’t support what you are saying”. It is like both parties are talking in two different alien languages with few common words. Does this, or a variation of this sound familiar to you?
If it does, then I suspect you are caught in, what I call as a “hubric cycle of denial”. What this means is that, all parties concerned fail to acknowledge and in extreme cases deny the other parties’ efforts and concerns. The product team (in this illustration) refuses to acknowledge the findings, and the research(er) team stands frustrated as they try to find alternative narratives that will drive their point across.
There is plenty written about why people find comfort in numbers (like this paper by Little & Lintz, or this one titled “Man as an intuitive statistician” by Peterson & Beach or why uncertainty causes stress by Peters et.al). But when all you have is a story, a narrative, what do you do? What should you do?
In other words, what do you do when all you have is a Hattori Hanzō against a Gatling Gun?
Answer: You recognize the threat and strategize.
A UX researcher finds themselves in an enviable position of manning the drawbridge over the proverbial moat. One needs to exhibit the acumen of comprehending the orders from inside the castle and heeding call for help from those stuck outside.
What this means is that the researcher needs to understand what the algorithm makers and the algorithm itself is saying. And to that effect, it is imperative that the researcher knows the language, those who speak the language and the history of the problem.
While the last bit - history of the problem is often shared with researchers in a sort of cliff notes approach with focus on the present context/background and why solving/answering this particular problem/question is of importance. I suspect this approach and acceptance of the research question/problem by the researcher is the root cause of the end stage Mexican stand-off.
Contrary to what Betty Williams suggests, [“There's no use talking about the problem unless you talk about the solution”], I believe it is important for a UX researcher to know the following:
Who recognized the question/problem
Why is this question/problem of importance
Does this question/problem mask some underlying question/problem
Do the people dealing with numbers have a hypothesis? Can you “picture” it? [As in envision a user to have similar thoughts/behavior for the suggested reasons]
Finding answers to these basic big header questions, will equip you or at least give you an insight into who and how the final insights will be consumed. Similar to how you word/frame your questions to the user/consumer, you can use this bit of information to frame/word your narrative to the internal stakeholder. Finding answers to these questions also aids in the researcher grasping the nuances of the language employed by the algorithm makers.
You may find that your responses start taking the shape of “The X number of people who were exhibiting <abc> behavior is because of _____” or some flavor or variant of this approach. My personal favorite, something that I learned dealing with Economists, is the following, “We acknowledge that the sample size is small but this <abc> behavior was noticed almost universally and has significant bearing on our current and future work because______”.
What these “number speak” styled responses do is two fold -
a) The audience is not translating it internally into a language they understand and are comfortable with.
b) It also primes the audience to focus on the behavior and not focus on the numbers.
TL;DR - Empathy begins at home. Empathy is not just a trait that is employed when dealing with users and consumers. It needs to be a default state of being as a researcher. And it needs to be exercised with internal stakeholders first.
The reason I mention empathy is precisely because when the engineering/product team is confronted with user behavior, what is taking place is essentially a showdown between the creators’ utopian vision and reality. And most often than not, reality often disappoints. And this is an upsetting, especially when the core objective of the UX research team is to explain to the creators the discordance between their vision and reality. So be gentle and more importantly be empathetic.
What I am saying may come across as being bit too “common-sensical”, but it is something that often gets overlooked.
In other things of interest, I recommend…
Watching “Racket Boys”, a K-Drama currently streaming on Netflix (every Monday, Tuesday around 2045hrs). A fun, funny, coming of age story.
“Sophie: Murder in West Cork” a riveting 3 episode documentary (Again on Netflix)
Listen to “Off Menu”, podcast by James Acaster and Ed Gamble where guests decide their ideal 3 course menu. Funny and engaging.
Also, listen to “Gossipmongers” by David Earl and Joe Wilkinson. Genuine laugh out loud which appeases my itch of participating in salacious gossipmongering.
If the latest season of “My Hero Academia” is going too slow for you, I strongly recommend you start reading the manga. There are tonnes of plot twists that you want to know about.
I recently (re)started watching “Succession” (currently streaming on Disney Hotstar) after having abandoned it in the middle of Ep.1. I don’t understand it. No character is likeable. It is not like Billions, but wants to be. All the characters in Succession seem to have a giant magnet next to their moral compass which makes them go crazy. And there is only so much craziness I can handle in today’s time.
Finished reading “Framers: Human Advantage in an Age of Technology and Turmoil” by Francis de Véricourt, Kenneth Cukier, and Viktor Mayer-Schönberger. Was a me-h read. Pretty much a Pop-science book whose entire premise is basically - “Try to look at a problem in more than one way”. Not sure who it is written for. Maybe people need to be told the obvious with examples, repeatedly.
I also read Leidy Klotz’s “Subtract”. Was long winded but fun. Some of his findings are geniunely interesting. I was especially surprised to find that there were no significant cultural departures in the manner in which people approached a problem. One would intuitively believe that problem solving approaches would have some cultural differences. Apparently not. Like I said, was fun and informative. Akin to taking a scenic route to a destination - but the scenery gets boring after a while but fun nonetheless.
Currently, reading “In the Miso Soup” by Ryu Murakami. So far so good. Will probably explain my recent and growing obsession with Korean and Japanese media in the next post and why you too should start reading and watching more things coming from Korea and Japan if you aren’t already.
Also if you have time on Thursday, do attend the fascinating Bochum workshop/talk series on “Why do people believe weird things” (1730hrs IST).
If you have suggestions on what to watch, read, listen, or things you would want me to talk about please let me know.
If you think you there are others who might like reading this, then please feel to share.
And if you think you might be interested in reading more about the things that interest me then do subscribe.