As an actual user researcher (qual, ABD, ~15 y.e.), I look at tools like these the way I look at no-code tools. They can really improve speed and lower barriers...but they're really, really context-dependent. Basically, if you're not a UXR, you're going to be asking the wrong questions and focusing on the wrong things; you've selected "fast" and "cheap" while leaving out "good." This is...usually...better than nothing, and often better than untrained PMs trying to "talk to customers." On the other hand, just as you'd think twice about deploying some no-code tool into production at scale, you should think twice about relying too heavily on surveys in general, much less "low-code research" tools. Corporations rely on them, usually because it makes people look busy and for the same reason that Oreos have lecithin and hydrogenated fat--fast and cheap, usually requirements for scale.
For a startup, it often makes a lot more sense to include "good." Use butter in your cookies. If you survey and A/B test your way to a product, you *will* build something that needs a comprehensive overhaul later, mostly because of assumption stacks. In the old days, this usually self-corrected the hard way by the time a corporation got listed. The problem I'm seeing more and more is that a well-funded startup keeps tacking on features, the product gets increasingly difficult to use, and then the CEO wonders why everyone's abandoning the product for a new, sleeker, much simpler competitor that is more closely modeled to actual user needs and mental models. (Let's just say Substack is doing this right now. The ship has sailed for Evernote.) By that time, calling in a (very expensive) fire team of qualified researchers may be too late. They'll tell you what's wrong down to the pixel, but you may not be able to react effectively. The worst part? You might have "AOL syndrome"--your numbers look good, but you don't realize it's because people have forgotten to cancel their subscription. And then they do.
Qual insights do need to scale--but you need to know exactly who and why. "Everybody" is still not an answer.
Appreciate the insights! Do you think trained user researchers can help set the company’s strategy around using tools like UserLeap so that everyone on the team who uses them does it “right”? Just like with Looker, non-BI/data scientists can pull numbers based on the structure and framework the trained professionals put in place.
Absolutely, and I'm seeing this often as an expansion of the UXR role. Another emerging biggie: a research repository, anything from a wiki to a spreadsheet, that non-specialists can consult on, say, emotional vs. rational messaging in a sales page. "We found 11/21 that lead customers prefer emotional appeals," for instance, with a link to the report.
Makes a lot of sense! Should probably edit the post to clarify: I definitely don’t think it’s going to be unstructured chaos, more UXR setting up the structure and giving more people the power to move quickly within it. Thanks!
Hi TW! I'm the Head of User Research here at UserLeap. Definitely hear your concerns. We've actually done a lot of work to built templates and best practices into the platform for non-researchers and work closely with a lot of our customers. I've seen many PMs and others generate valuable insights through contextual microsurveys with UserLeap. And by the way, we work with lots of User Research teams as well!
As an actual user researcher (qual, ABD, ~15 y.e.), I look at tools like these the way I look at no-code tools. They can really improve speed and lower barriers...but they're really, really context-dependent. Basically, if you're not a UXR, you're going to be asking the wrong questions and focusing on the wrong things; you've selected "fast" and "cheap" while leaving out "good." This is...usually...better than nothing, and often better than untrained PMs trying to "talk to customers." On the other hand, just as you'd think twice about deploying some no-code tool into production at scale, you should think twice about relying too heavily on surveys in general, much less "low-code research" tools. Corporations rely on them, usually because it makes people look busy and for the same reason that Oreos have lecithin and hydrogenated fat--fast and cheap, usually requirements for scale.
For a startup, it often makes a lot more sense to include "good." Use butter in your cookies. If you survey and A/B test your way to a product, you *will* build something that needs a comprehensive overhaul later, mostly because of assumption stacks. In the old days, this usually self-corrected the hard way by the time a corporation got listed. The problem I'm seeing more and more is that a well-funded startup keeps tacking on features, the product gets increasingly difficult to use, and then the CEO wonders why everyone's abandoning the product for a new, sleeker, much simpler competitor that is more closely modeled to actual user needs and mental models. (Let's just say Substack is doing this right now. The ship has sailed for Evernote.) By that time, calling in a (very expensive) fire team of qualified researchers may be too late. They'll tell you what's wrong down to the pixel, but you may not be able to react effectively. The worst part? You might have "AOL syndrome"--your numbers look good, but you don't realize it's because people have forgotten to cancel their subscription. And then they do.
Qual insights do need to scale--but you need to know exactly who and why. "Everybody" is still not an answer.
Appreciate the insights! Do you think trained user researchers can help set the company’s strategy around using tools like UserLeap so that everyone on the team who uses them does it “right”? Just like with Looker, non-BI/data scientists can pull numbers based on the structure and framework the trained professionals put in place.
Absolutely, and I'm seeing this often as an expansion of the UXR role. Another emerging biggie: a research repository, anything from a wiki to a spreadsheet, that non-specialists can consult on, say, emotional vs. rational messaging in a sales page. "We found 11/21 that lead customers prefer emotional appeals," for instance, with a link to the report.
Makes a lot of sense! Should probably edit the post to clarify: I definitely don’t think it’s going to be unstructured chaos, more UXR setting up the structure and giving more people the power to move quickly within it. Thanks!
Hi TW! I'm the Head of User Research here at UserLeap. Definitely hear your concerns. We've actually done a lot of work to built templates and best practices into the platform for non-researchers and work closely with a lot of our customers. I've seen many PMs and others generate valuable insights through contextual microsurveys with UserLeap. And by the way, we work with lots of User Research teams as well!