Do you support providing more foreign aid and reducing military spending?, Ambiguous term, 'foreign aid' is not defined, Leading question, pushes respondents toward a particular answer, Recall difficulty, respondents cannot remember past spending levels, Double-barrelled question, combines two separate policy issues into one, How often do you usually visit a healthcare provider?, Social desirability bias, people want to appear healthy, Double-barrelled, asks about both frequency and type of provider, Incomplete wording, doesn't specify the age of the respondent, Vague frequency adverb, 'usually' means different things to different respondents, Don't you think governments should not be so uncooperative with international climate agreements?, Question order effec, depends on what was asked before, Lack of a middle option, respondents are forced to choose a side, Multiple negatives, the phrasing makes it extremely difficult to determine what 'agree' means, Social desirability, climate concern is seen as socially valued, How would you rate the quality of education in your country — would you say it is rewarding but stressful, not very rewarding but not stressful, or not rewarding at all?, The scale is not unidimensional, it mixes two different concepts (reward and stress), Leading question, pushes respondents toward negativity, Too few response options, should have at least 7 points, Recall problem, respondents can't remember their education experience, The extent to which it has been determined by authorities that restrictions pertaining to cross-border movement of persons should be implemented was thought by you to be adequate or inadequate?, Social desirability — immigration is a sensitive topic, Excessive length and grammatical complexity — passive voice, convoluted syntax overwhelm respondents, Double-barrelled — asks about both border restrictions and adequacy, Unrealistic — asks about a future policy, Do you think the government should forbid extremist political groups to hold public rallies?, Social desirability — people don't want to appear intolerant, Multiple negatives — 'not allow' creates a confusing double negative with 'disagree', Leading question — the word 'extremist' biases the response, Forbid/allow asymmetry — 'not allow' produces different responses than 'forbid' even though logically equivalent, Rate your satisfaction with public transport on a scale from -5 (very dissatisfied) to +5 (very satisfied)., Bipolar numeric scale — negative numbers shift the frame of reference and produce different distributions than a 0–10 unipolar scale, Too many response options — 11 points is too fine-grained, Missing middle option — zero is ambiguous, Social desirability — people want to appear positive about their city, A 12-item response scale uses only verbal labels at the endpoints ("not at all satisfied" and "completely satisfied") with no numbers shown to respondents between them., Social desirability — satisfaction is always over-reported, Too few response options — 12 points is actually fine, Acquiescence bias — semantic differentials don't involve agree/disagree, Inadequate scale labelling — all numeric labels should be shown to respondents, with explicit verbal anchors at endpoints, Do you support the policies of the IMF?, Leading — suggests support is the expected position, Assumes knowledge — respondents may not know what the IMF or its policies are, Double-barrelled — asks about two separate policies, Social desirability — pressure to agree with global institutions, Don’t you agree that developed countries should help poorer countries?, Double-barrelled — asks about two countries, Leading — wording pushes toward agreement, Ambiguous — “help” is unclear, Excessive recall period, How satisfied are you with your country’s schools and hospitals?, Leading — assumes they are satisfied, Double-barrelled — combines two different services, Assumes knowledge of both schools and hospitals, Double negative, Do you disagree that globalization does not help poor countries?, Double negative — confusing logic, Double-barrelled - it asks two things at the same time, Leading — pushing respondents to assume globalization helps, Loaded / ambiguos wording , Don’t you agree that climate change is the biggest global threat?, Double-barrelled — refers to multiple threats, Leading — wording pushes toward agreement, Ambiguous — “biggest” is unclear, Assumes knowledge, Do you support UN climate initiatives?, Assumes knowledge — requires awareness of UN initiatives, Leading — wording pushes toward agreement, Double negative, Double-barrelled, How satisfied are you with your university experience — would you say you are very satisfied, satisfied, dissatisfied, or very dissatisfied?, Missing middle option — respondents without clear opinions are forced to choose a side, Leading question — assumes students should feel satisfied, Recency effect — students always choose the last option, Double-barrelled question — asks about two things at once, How would you rate the president's handling of the economy — would you say you strongly approve, approve, disapprove, or strongly disapprove?, Recency effect — 'strongly disapprove' is always chosen because it's last, Missing middle option — no neutral point forces respondents without opinions into a direction, increasing error, Social desirability — respondents don't want to criticise the president, Leading question — 'handling' implies the president is actively managing, How would you rate the quality of healthcare in your country: excellent, very good, good, fair, or poor?, Vague quantifier — 'fair' is ambiguous, Too many response options — five points is excessive for this topic, Social desirability — people want to appear grateful for healthcare, Unbalanced scale — three positive options but only two negative ones, biasing responses toward the positive end, Do you support the government's new artificial intelligence policy — strongly support, support, neither support nor oppose, oppose, or strongly oppose?, Missing middle option — respondents cannot express neutrality, Missing "Don't Know" option — respondents without knowledge may give inaccurate answers, Leading question — encourages support, Social desirability bias — respondents want to appear informed, On a scale from 0 to 25, "How satisfied are you with your life?", Missing verbal labels — respondents may interpret numbers differently, Recency effect, Missing "Don't Know" option, Double-barrelled question, How many hours do you study per week — 0–5 hours, 5–10 hours, 10–15 hours, or more than 15 hours?, Missing "Don't Know" option, Overlapping categories — some responses fit more than one category, Leading question, Recency effect.

Tutorial 2. Survey Question Problems

by
Print
Embed

Leaderboard

Visual style

Options

Switch template

)
Continue editing: ?