Internal Validity: The degree to which a study establishes a trustworthy cause-and-effect relationship between treatment and outcome, free from confounding variables.
- Was the study experimental or observational?
- Were participants randomly assigned to groups?
- Are there confounding variables?
- Is there evidence of other biases?
Observational study - participants were not randomly assigned to drink coffee
No random assignment - coffee drinkers vs. non-drinkers were pre-existing groups
Many potential confounders: lifestyle, age, education, health consciousness
Poor internal validity - cannot establish causation
Confounding Variables: Multiple potential confounders
Causal Inference: NOT POSSIBLE
The study has poor internal validity. It's an observational study with no random assignment, creating multiple confounding variables. Coffee drinkers may differ from non-drinkers in many ways beyond coffee consumption, making it impossible to establish causation.
• Selection bias: Non-random group assignment
• Confounding: Third variables affecting both exposure and outcome
• History effects: External events during study
• Maturation: Natural changes over time
Students from elite private school - may not represent general student population
Elite private school environment differs from typical public schools
Resources, student motivation, teacher quality, parental involvement may differ
Limited external validity due to specific population and setting
✓ High: Randomized controlled trial
✓ Proper experimental design
✓ Can infer causation
✗ Limited: Elite private school
✗ May not generalize
✗ Restricted population
External Validity: LOW (limited population)
Generalizability: LIMITED to similar contexts
The study has high internal validity due to randomization, but low external validity. Results may not generalize to students in different educational contexts due to the specific population (elite private school) and setting.
• Population characteristics: Age, socioeconomic status, education level
• Setting: Geographic location, institutional context
• Time period: Historical context and changing conditions
• Intervention delivery: Who delivers and how it's implemented
Selection Bias: Systematic error due to the way participants are selected for the study, leading to a sample that is not representative of the target population.
Advertisement targeting people with severe symptoms
Only severe cases recruited, not representative of general population
Severe symptoms naturally tend to improve over time
Voluntary response bias, expectation effects
Additional Biases: Regression to mean, expectation effects
Impact: Overestimated effectiveness
Major bias: Selection bias from recruiting only severe cases. Additional biases: regression to the mean (severe symptoms naturally improve), expectation effects. Results may overestimate true drug effectiveness.
| Bias Type | Description | Impact |
|---|---|---|
| Selection Bias | Non-representative sample | Results don't generalize |
| Response Bias | Systematic response differences | Data inaccuracies |
| Confirmation Bias | Seeking confirming evidence | Interpretation errors |
| Survivorship Bias | Missing dropout data | Overestimated effects |
Focus: Did the treatment cause the observed effect?
Requires: Random assignment, control of confounders
Focus: Do results apply beyond the study context?
Requires: Representative sampling, appropriate setting
Information Bias: Measurement errors
Confounding: Third variables affecting results
Reporting Bias: Selective outcome reporting
Confounding Variable: A variable that influences both the explanatory and response variables, creating a spurious association between them.
Breakfast consumption → Academic performance
Variables that affect both breakfast habits and academic performance
Confounders may explain the observed association
Causal conclusion is not supported by observational design
Validity Issue: No random assignment
Conclusion: Association, not causation
Many confounding variables exist: sleep schedules, parental involvement, socioeconomic status, general health consciousness, time management skills. The study cannot establish causation from an observational design with confounding variables.
• Sleep patterns: Early risers who eat breakfast may sleep better
• Parental involvement: Parents who ensure breakfast may also help academically
• Socioeconomic status: Higher SES families more likely to eat breakfast
• Self-discipline: Organized students do both breakfast and homework
Comprehensive Study Evaluation: Systematic assessment of all aspects of study design to identify strengths, weaknesses, and validity threats.
Convenience sampling of gym members → Poor external validity
Observational cross-sectional study → Poor internal validity
Health consciousness, social connections, income, motivation
Both internal and external validity are compromised
External Validity: POOR (convenience sample of gym members)
Bias Sources: Selection bias, confounding variables
Conclusion: Limited interpretability
Major issues: Convenience sampling of gym members limits generalizability; observational design prevents causal inference; multiple confounding variables present. Improvements: Random sampling from general population, longitudinal design, control for confounders.
• Sampling bias: Gym members not representative of general population
• Temporal ambiguity: Cross-sectional cannot establish direction of causation
• Self-report bias: Survey data may be inaccurate
• Healthy user bias: Gym members may have healthier overall lifestyles
🔍 Definitions of Key Concepts
⚖️ Core Rules and Principles
⚙️ Step-by-Step Methods for Validity Assessment
- Study Design: Identify if experimental or observational
- Sampling Method: Determine if random sampling was used
- Assignment Method: Check for random assignment
- Confounding Variables: Identify potential confounders
- Measurement: Assess for measurement bias
- Analysis: Evaluate statistical methods and interpretation
- Conclusion: Determine validity of inferences
🧪 Examples: From Simple to Advanced
💡 Tips, Tricks, and Common Pitfalls
📝 Key Notes for Memorization
📋 Validation Checklist
- Was the study design appropriate for the research question?
- Were participants randomly assigned (for internal validity)?
- Was the sample randomly selected (for external validity)?
- Are there potential confounding variables?
- How were variables measured?
- Are the conclusions supported by the data?
- What are the limitations of the study?
- How do the results compare to other studies?
Questions & Answers
Question: How can I tell if a study has confounding variables just by reading about it?
Answer: Look for these red flags:
- Observational studies: Non-random assignment often means confounding is possible
- Specific populations: Studies on unusual groups (college students, volunteers) may have selection bias
- Strong correlations: Extremely high correlations in observational studies suggest confounding
- Direction of causation: Unclear which variable comes first
Questions to ask: "What else could explain this relationship?" and "Are there variables that affect both the predictor and outcome?"
Example: If a study shows that people who buy organic food have better health, confounders might include income, education, health consciousness, etc.
Question: What's the difference between bias and confounding? Aren't they the same thing?
Answer: They're related but different concepts:
- Bias: Any systematic error that deviates from the truth (includes selection bias, measurement bias, etc.)
- Confounding: A specific type of bias where a third variable affects both exposure and outcome
Analogy: Bias is like having a faulty ruler that always gives wrong measurements. Confounding is like trying to measure height but the person is standing on a box - the box (confounder) affects both the measurement and the true height.
Relationship: Confounding is a type of bias, but bias is a broader concept that includes many other types of systematic errors.
Question: Can a study have high internal validity but low external validity? Give an example.
Answer: Yes, this is very common! Here's an example:
- Study: Randomized controlled trial of a new medication with strict inclusion criteria (healthy young adults only)
- Internal validity: High - random assignment, controlled environment, clear causal inference
- External validity: Low - results may not apply to elderly, sick, or diverse populations
Real-world example: Many clinical trials have high internal validity but limited external validity because they study very specific populations under controlled conditions.
Trade-off: Researchers often sacrifice external validity to achieve higher internal validity, creating a tension in study design.