Survey 101:Choosing Right Questions Types

Creating questionnaires is one of the most fundamental and widely used methods in user experience research. Almost everyone has encountered some form of research questionnaire while growing up. A questionnaire is simply a list of questions, with participants providing direct responses to generate data.

This research method offers a quick, cost-effective way to gather both quantitative and qualitative data, allowing researchers to identify behavioral patterns, assess the feasibility of a design project, and even screen ideal candidates for further studies. However, designing an effective research questionnaire is far more challenging than it appears. A well-crafted questionnaire can yield rich and meaningful insights with minimal resources, while a poorly designed one can waste time and effort without generating useful information.

In Survey 101 series, I’ll be sharing practical advice on questionnaire design, distribution, and analysis in a series of articles aimed at helping you improve your research methods. If you’re a student new to UX research, this will be a great place to start!

In today’s post, I’ll focus on one of the most fundamental questions in survey design: What question types should I use in my questionnaire?

To put it simply, open-ended questions require participants to type their answers, while closed-ended questions allow them to simply click on a response. It’s clear that open-ended questions take longer to complete, which contradicts one of the key principles of user-friendly questionnaires I discussed in my previous post. The longer a questionnaire takes, the more likely participants are to experience fatigue, which can lower the quality of the data collected. In general, I don’t recommend using open-ended questions unless absolutely necessary—only for cases where a question truly cannot be answered through predefined choices.

I’ve noticed that some people prefer using open-ended questions simply out of convenience. Yes, setting up closed-ended questions requires more effort because you have to carefully list out all options and refine the words. However, as I’ve said before, designing a questionnaire is a kind of designing user experience. If you don’t offer a good experience, how can you expect users to provide their valuable data that you’re looking for?

Another major issue with open-ended questions is data analysis. Even if you receive many responses, manually sorting through and interpreting open-ended answers becomes a nightmare once your sample size exceeds 100 participants—unless you have strong programming skills or a dedicated team. In contrast, closed-ended questions allow for easy data manipulation using tools like Excel, making it much simpler to generate clear, persuasive visualized insights for presentations or reports. I know you need something to persuade stakeholders, so ensure your questionnaire can provide you these cool numbers.

That being said, closed-ended questions also have their limitations. They may not always capture unique or uncommon experiences that some participants may have. Even if you try to anticipate all possible responses, listing too many options can overwhelm users. In these cases, a hybrid approach works best: include an "Other" option within a closed-ended question, allowing participants to type their own answer if none of the predefined choices apply to them. This balances efficiency and flexibility—minimizing response time while ensuring inclusivity. (Of course, this doesn’t mean you should add an “Other” option to every question. It should be used selectively based on the nature of the question.)

One scenario where open-ended questions are necessary is when collecting participant contact information for follow-up research. At the end of my questionnaires, I typically include an open-ended question thanking respondents for their time and asking if they’d like to participate in the next stage of user interviews. This is usually the only typed-response question in my surveys. As I mentioned earlier, another important function of a questionnaire is to screen candidates for deeper research. While questionnaires provide quick quantitative insights, they don’t always offer the depth needed for truly unique and valuable discoveries—you need to get prepared for the next step.

Most people are comfortable setting up choice questions in their questionnaires. These question types are quick for participants to complete, requiring only a few taps on the screen. They also make data analysis incredibly easy—many questionnaire platforms even provide built-in visualization tools, eliminating the need to manually process data in Excel.

Technically, it’s possible to design an entire questionnaire using only single- or multiple-choice questions, but that approach can feel a bit dull and repetitive. More importantly, different types of questions can enhance data quality, improve user engagement, and make the questionnaire more visually appealing and structured.

I’ll introduce a few alternatives to traditional multiple-choice questions that are particularly effective in certain situations. These alternatives streamline responses, enhance clarity, and make the questionnaire more compact and organized—for better participant experience.

Sorting questions are useful when you want to assess priorities, preferences, or rankings among multiple options. Instead of setting up two separate multiple-choice questions to identify the most and least preferred choices, a single sorting question allows participants to rank all the given options in order of preference or importance. This makes data collection more efficient while providing a clearer picture of participant tendencies.

However, sorting questions also have their drawbacks. If a participant has similar opinions about two or more options, they are still forced to rank them in a specific order. This may introduce artificial distinctions that don’t truly reflect the participant’s feelings, potentially affecting the reliability of the final data.

If you choose to add a sorting question into your questionnaire, please keep the list of options manageable and understandable—too many items can make the ranking process tedious, leading to longer thinking time and bias data.

Evaluation scale questions are widely used to assess participants’ perceptions, opinions, or emotions regarding a given topic. They allow for nuanced responses based on scales, making them particularly effective in capturing degrees of agreement, satisfaction, or importance.

One of the biggest advantages of evaluation scale questions is that they translate responses into numerical data, which enhances persuasiveness and clarity in reporting findings.

Using a slider-based scale instead of traditional numbered options (such as a 1-5 or 1-10 scale) can improve data accuracy for even more precise and user-friendly responses. However, when using sliders, it’s essential to ensure clarity in scale interpretation so that participants understand what the endpoints and middle values represent.

When evaluating existing products to understand user perceptions, identify problems, and uncover design opportunities, matrix questions can be highly effective. Instead of creating multiple consecutive multiple-choice questions, a matrix format condenses multiple related questions into a structured table, allowing participants to rate different attributes using the same scale.

One of the key advantages of matrix questions is that they reduce survey time and minimize participant fatigue. By grouping similar questions together, respondents can evaluate multiple aspects efficiently without having to repeatedly read similar question stems. This streamlined approach improves the user experience and enhances data consistency.

However, matrix questions should be kept concise to avoid overwhelming participants. If the table contains too many rows, respondents may rush through answers or select the same response across all rows without careful consideration. To maintain accuracy and engagement, it’s important to limit the length of matrix questions and ensure that all attributes being evaluated are clearly defined and distinct.

Overall, I recommend primarily using closed-ended questions when designing your questionnaire, while strategically incorporating special question types to increase flexibility and optimize the survey length. Open-ended questions can be incredibly valuable for gathering detailed, specific insights, but they should be used sparingly to prevent participant fatigue and ensure efficient data analysis.

Of course, every project is unique, and my recommendations are not absolute rules. Questionnaire design should always be adapted to fit the needs of the research. If you find that my advice doesn’t fully apply to your situation, feel free to experiment and get creative! After all, no one understands your project better than you do.

If you found this article helpful, please don’t hesitate to share it with your friends, like it, or subscribe to my channel, "Curious Loom." Your support lets me know my work is making a difference. Thank you so much!