On Properly Measuring User Experience

On Properly Measuring User Experience

Sep 27, 2023

Dear readers & friends,

The third, and final, article of my series on UX measurement, “Seven heuristics for identifying proper UX instruments and metrics,” has been published on the ACM Interactions blog.

It all started with a discussion with a colleague on whether conversion rates are a reliable measure of user experience. That one quickly turned into an article dealing with not only CR, but also AOV (average order value), NPS (Net Promoter Score), and a bunch of other measures that are—to the surprise of some—not UX metrics.

After that first article was published, some people asked, “if all of those are not really suitable, then how can we measure UX?” Thus, the second part of the series saw the light of day—published in ACM Interactions magazine. It explains that to properly measure UX, you need what’s called an instrument. One example is UEQ, the User Experience Questionnaire. In some cases, user interactions and web analytics data then can serve as predictors for such instruments.

However, there are plenty of instruments out there that supposedly measure UX—often promoted by companies more concerned with sales and marketing than accuracy and reliability—but don’t really. So, how can we identify the good ones? That’s exactly what my latest article helps with. It gives you 7 heuristics to easily identify metrics that actually reliably measure UX and shows how to apply them in 2 short case studies (UEQ—good; and QXscore—bad).

Have fun reading and please feel free to let me know your thoughts on this. 💭

A researcher in a thinking pose, surrounded by a range of devices, diagrams, and charts.

And, as usual, if my own articles are not enough reading material for you …

Reading Recommendations

📖 You are not a Product Manager by Jakub Brabec challenges what is nowadays often considered to be product management: “If your main agenda is to tell designers and tech leads how to do their jobs and then stick to their ass until it’s done, you are not a PM. If you mainly run standups, sprint planning, and refinement, you are not a PM. You are a product designer, solution architect, or a scrum master. Not a PM.”

📖 Relatedly, in Evolution of the Product Manager, Ellen Chisa argues that we need better education to develop the discipline, and gives an overview of how such education might look like.

Tool Recommendations

🛠 All-in-one UX Research Dashboard in Notion is the one thing I wish I would’ve had in my previous job at C&A, where our team was in charge of running the UX research program. It’s a powerful automated template that lets you plan, organize, and track research projects in one place rather than having to fiddle with 10 different tables on 5 different Confluence pages. Given the amount of time it would have saved me in that job, ~$20 is ridiculously cheap in my opinion.


Enjoy reading and until next time. As always, I welcome your feedback and comments on my articles and this newsletter. Simply reply to this email to share your thoughts.

Cheers,
Max


This is an issue of my newsletter Tales of Design & User Experience (& Other Stuff)—abbreviated ToDUX—that has been cross-posted here for archival reasons. If you want to get my latest takes and writing delivered straight to your inbox, you can subscribe at maxspeicher.com/newsletter or simply follow me here on Buy Me A Coffee. 🗞

Enjoy this post?

Buy Max(imilian) Speicher a coffee

More from Max(imilian) Speicher