As an actual user researcher (qual, ABD, ~15 y.e.), I look at tools like these the way I look at no-code tools. They can really improve speed and lower barriers...but they're really, really context-dependent. Basically, if you're not a UXR, you're going to be asking the wrong questions and focusing on the wrong things; you've selected "fast" and "cheap" while leaving out "good." This is...usually...better than nothing, and often better than untrained PMs trying to "talk to customers." On the other hand, just as you'd think twice about deploying some no-code tool into production at scale, you should think twice about relying too heavily on surveys in general, much less "low-code research" tools. Corporations rely on them, usually because it makes people look busy and for the same reason that Oreos have lecithin and hydrogenated fat--fast and cheap, usually requirements for scale.

For a startup, it often makes a lot more sense to include "good." Use butter in your cookies. If you survey and A/B test your way to a product, you *will* build something that needs a comprehensive overhaul later, mostly because of assumption stacks. In the old days, this usually self-corrected the hard way by the time a corporation got listed. The problem I'm seeing more and more is that a well-funded startup keeps tacking on features, the product gets increasingly difficult to use, and then the CEO wonders why everyone's abandoning the product for a new, sleeker, much simpler competitor that is more closely modeled to actual user needs and mental models. (Let's just say Substack is doing this right now. The ship has sailed for Evernote.) By that time, calling in a (very expensive) fire team of qualified researchers may be too late. They'll tell you what's wrong down to the pixel, but you may not be able to react effectively. The worst part? You might have "AOL syndrome"--your numbers look good, but you don't realize it's because people have forgotten to cancel their subscription. And then they do.

Qual insights do need to scale--but you need to know exactly who and why. "Everybody" is still not an answer.

Expand full comment