What do I use
Psychology and behavioural science
Scale development, experiments, questionnaire studies, intervention trials.
Your question, and what answers it
Cronbach's alpha with its interval and the item-total correlations, which is where a missed reverse-coding shows up.
In ZStat: open your study, go to Analysis, and choose reliability.
Exploratory factor analysis when you are finding out, confirmatory when you are testing a structure you already claimed.
In ZStat: open your study, go to Analysis, and choose efa.
Run this BEFORE comparing group means. If scalar invariance fails, a difference in scores is partly a difference in how the items were read.
In ZStat: open your study, go to Analysis, and choose measurement invariance.
Repeated measures when the same people were measured twice, with sphericity checked and corrected.
In ZStat: open your study, go to Analysis, and choose repeated measures anova.
Bootstrapped, and only causal if the mediator was measured between the two in time.
In ZStat: open your study, go to Analysis, and choose mediation.
With the interaction probed rather than asserted.
In ZStat: open your study, go to Analysis, and choose moderation.
Item response theory. Wants several hundred respondents before the discrimination estimates mean much.
In ZStat: open your study, go to Analysis, and choose irt 2pl.
What goes wrong in psychology and behavioural science
- Reporting alpha as evidence of unidimensionality. It rises with the number of items and a twenty-item scale measuring two things easily exceeds .90.
- Comparing group means on a scale that has not passed scalar invariance.
- A median split on a continuous variable, which costs about a third of your sample.