Lee, A.S., and Baskerville, R.L. “Generalizing generalizability in information systems research,” Information Systems Research (14:3), Sep 2003, pp 221-243.

Key passage #1:

An increase in sample size is beneficial, but the benefits take the form of improved reliability of the sampling procedure, rather than improved generalizability of a sample to its population. [226]

Bigger sample size doesn’t mean better generalizability to the entire population. A sample is a sample, not matter how big (until N = pop). It just means your sampling procedure is more likely to produce repeatable results. A researcher can take as much of a stab at generalization from one case study as from a quantitative analysis of a big-N sample. Both methods could make the right generalization; both could guess wrong. It’s up to later researchers to disprove that generalization.

Think of it as a language clarification: Saying you have a large sample doesn’t prove that your results will more likely apply to the entire population than the results of a smaller-sample study. It just says that if someone else applies your methods, it’s more likely they will get similar results. Nothing inductive can tell us about the items we haven’t surveyed yet. “Therefore, a larger sample size does increase generalizability, but it is the generalizability of a sample to other samples, not to the population” [227].

Key passage #2:

Geertz states the following about both theory and generalizability in anthropological studies about culture (Geertz 1973, pp. 25-26): “The essential task of theory building here is not to codify abstract regularities but to make thick description possible; not to generalize across cases but to generalize within them.”  [231]

The positivist wants to be able to extrapolate to the universe; the interpretivist is dealing with a different, sometimes not extrapolable beast: the meanings that exist only in the context of their culture. The interpretivist can find as much value in learning the general principles of meaning within a group, even if that information doesn’t generalize to the meanings other groups construct.

Overall, Lee and Baskerville offer a liberating message: recognizing the limits of a proper understanding of the term “generalizability” gives researchers more freedom to pursue interesting and useful ideas without worrying quite so much about proving that their results will apply everywhere to everyone.

Advertisements