Generalizability: The extent to which findings from a sample can be applied to a larger population. It depends on how the sample was selected.
- Identify the sampling method used
- Determine the sampling frame
- Compare sample characteristics to target population
- Assess representativeness
500 college students at one university
Students enrolled at this particular university
Likely intended to generalize to all college students
Results may not apply to all college students due to university-specific factors
Limited generalizability: May not represent all college students
Factor affecting generalizability: University-specific characteristics
Results can be generalized to students at this specific university. Generalizability to all college students is limited due to potential differences in academic rigor, student demographics, and institutional factors.
• Sampling method: Random sampling increases generalizability
• Sample size: Larger samples can improve precision
• Sampling frame: Should match target population
• Geographic location: Local factors may limit generalizability
Randomized controlled trial with treatment and control groups
Random assignment balances potential confounding variables
Placebo control provides baseline for comparison
Yes, causation can be inferred due to proper experimental design
Generalizable to: Similar patients
Requires: Random sampling
Inferred: Drug causes improvement
Requires: Random assignment
Supported by: Random assignment, control group
Conclusion: Drug likely caused the improvement
Yes, causation can be inferred. The random assignment balances confounding variables across groups, and the control group provides a baseline for comparison. The significant difference in outcomes is likely due to the treatment.
• Random assignment: Essential for balancing confounding variables
• Control group: Provides baseline for comparison
• Manipulation: Researcher actively applies treatment
• Temporal sequence: Treatment precedes outcome measurement
Scope of Inference: The combination of population generalizability and causal inference that determines what conclusions can be drawn from a study.
Random sampling from high school students nationwide → Good for generalizability
Random assignment to teaching methods → Good for causal inference
Results can be generalized to all high school students nationally
Method A causes better outcomes than Method B for these students
Causal Inference: YES (Method A superior to Method B)
Scope: Full scope of inference achieved
Both population inference and causal inference are possible. The random sampling allows generalization to all national high school students, and the random assignment allows causal inference that Method A is more effective than Method B.
| Design Feature | Population Inference | Causal Inference |
|---|---|---|
| Random Sampling + Random Assignment | ✓ Possible | ✓ Possible |
| Random Sampling Only | ✓ Possible | ✗ Not Possible |
| Random Assignment Only | ✗ Limited | ✓ Possible |
Requires: Random sampling from target population
Concern: External validity (to whom do results apply?)
Requires: Random assignment to treatment groups
Concern: Internal validity (is the relationship causal?)
Factors: Sampling method, sample representativeness, study context
Limitation: Even valid studies may have limited generalizability
Observational Study: A study where researchers observe subjects without manipulating variables, preventing causal inference due to potential confounding variables.
Observational study (survey) with no manipulation of variables
No random assignment - people chose their own tea consumption
No causal inference possible due to confounding variables
Depends on how the 10,000 people were selected
Population Inference: Depends on sampling method
Conclusion: Association exists, causation uncertain
Causation cannot be inferred because this is an observational study with no random assignment. Generalizability depends on the sampling method used to select the 10,000 people. The study shows association but not causation.
• Confounding variables: Other factors may explain the association
• No causal inference: Cannot establish cause-and-effect relationships
• Self-selection bias: Tea drinkers may differ from non-drinkers in many ways
• Alternative explanations: Healthy lifestyle factors may be responsible
Scope of Inference Evaluation: A systematic assessment of what conclusions can be drawn based on the study's design features and limitations.
Causal Inference: POSSIBLE (due to random assignment)
Overall Scope: Partial inference - causal but limited generalizability
The study allows causal inference (the new method likely caused the improvement) due to random assignment. However, population inference is limited to students similar to those at this specific school due to convenience sampling.
• Convenience vs. Rigor: Convenience sampling makes studies easier but limits generalizability
• Internal vs. External Validity: Can have strong internal validity (causal inference) but weak external validity (generalizability)
• Practical Constraints: Real-world limitations often affect optimal design choices
🔍 Definitions of Key Concepts
⚖️ Core Rules and Principles
⚙️ Step-by-Step Methods for Inference Evaluation
- Identify the sampling method: Was random sampling used?
- Identify the assignment method: Was random assignment used?
- Evaluate population inference: Can results be generalized?
- Evaluate causal inference: Can causation be concluded?
- Determine scope of inference: What conclusions are valid?
- Consider limitations: What factors might restrict inference?
🧪 Examples: From Simple to Advanced
💡 Tips, Tricks, and Common Pitfalls
📝 Key Notes for Memorization
📋 Inference Decision Matrix
| Study Design | Random Sampling | Random Assignment | Population Inference | Causal Inference |
|---|---|---|---|---|
| Survey (Random) | ✓ | ✗ | ✓ | ✗ |
| Experiment (Random) | ✗ | ✓ | ✗ (limited) | ✓ |
| Complete RCT | ✓ | ✓ | ✓ | ✓ |
| Observational | ✗ | ✗ | ✗ (limited) | ✗ |
Questions & Answers
Question: I'm confused about when a study can have both population and causal inference. Can you explain the difference?
Answer: These are two different types of inference that require different design features:
- Population Inference: "To whom do these results apply?" Requires random sampling from the target population
- Causal Inference: "Does X cause Y?" Requires random assignment to treatment groups
Analogy: Population inference is like asking "does this work for other people?" Causal inference is like asking "did the treatment actually cause the effect?"
Example: A study with random sampling from all high school students AND random assignment to teaching methods can support both types of inference.
Question: Why can't we infer causation from observational studies even if we find a strong association?
Answer: Observational studies have a fundamental problem called "confounding variables":
- Self-selection: People choose their own behaviors/exposures
- Confounding: Other variables might explain the association
- No manipulation: Researchers don't control the variable of interest
- Alternative explanations: Third variables might cause both the exposure and outcome
Example: People who exercise more might live longer, but maybe healthy people are more likely to exercise AND live longer, not because exercise causes longevity.
Result: We can only say there's an association, not causation.
Question: Can a study have poor population inference but good causal inference? Give an example.
Answer: Yes, absolutely! This is very common in clinical trials:
- Example: A drug trial recruits patients from a single hospital through convenience sampling, but randomly assigns them to treatment or control groups
- Causal inference: Excellent - random assignment allows strong causal conclusions
- Population inference: Poor - results may not apply to patients elsewhere or with different characteristics
Result: Researchers can confidently say "the drug causes improvement" but only for patients similar to those in the study.
Real-world: Many medical studies have strong internal validity (causal inference) but limited external validity (population inference).