Solved Exercises on Generalizability and Scope of Inference in Grade 10 Statistics

Master generalizability and scope of inference: population inference, causal inference, and study design implications through these 5 detailed exercises with comprehensive explanations.

Solution: Exercises 1 to 3
1 Population Inference
Exercise 1
A researcher conducts a survey of 500 college students at a university to estimate the average amount of time spent studying per week. The sample mean is 15 hours. To what population can these results be generalized? What factors affect the generalizability?
Definition:

Generalizability: The extent to which findings from a sample can be applied to a larger population. It depends on how the sample was selected.

Generalizability Assessment:
  1. Identify the sampling method used
  2. Determine the sampling frame
  3. Compare sample characteristics to target population
  4. Assess representativeness
Step 1: Identify the sample

500 college students at one university

Step 2: Determine the sampling frame

Students enrolled at this particular university

Step 3: Assess the target population

Likely intended to generalize to all college students

Step 4: Evaluate generalizability

Results may not apply to all college students due to university-specific factors

Sample: 500 students at one university
↓
Sampling Frame: University students
↓
Limited Generalizability to: Similar universities
Appropriate population: Students at this university
Limited generalizability: May not represent all college students
Factor affecting generalizability: University-specific characteristics
Solution:

Results can be generalized to students at this specific university. Generalizability to all college students is limited due to potential differences in academic rigor, student demographics, and institutional factors.

Factors Affecting Generalizability:

• Sampling method: Random sampling increases generalizability

• Sample size: Larger samples can improve precision

• Sampling frame: Should match target population

• Geographic location: Local factors may limit generalizability

Tip: Always consider the sampling method when evaluating generalizability.
Key Note: Larger samples don't necessarily improve generalizability if the sample is biased.
2 Causal Inference
Exercise 2
A study randomly assigns 200 patients to receive either a new drug or a placebo. The group receiving the new drug shows significantly greater improvement in symptoms. Can causation be inferred? What aspects of the study design support this conclusion?
📊 Causal Inference Criteria
To infer causation, a study must meet specific design requirements that eliminate alternative explanations.
Step 1: Identify the study design

Randomized controlled trial with treatment and control groups

Step 2: Evaluate random assignment

Random assignment balances potential confounding variables

Step 3: Assess control group

Placebo control provides baseline for comparison

Step 4: Determine causal inference

Yes, causation can be inferred due to proper experimental design

Population Inference

Generalizable to: Similar patients

Requires: Random sampling

Causal Inference

Inferred: Drug causes improvement

Requires: Random assignment

Causal inference: YES
Supported by: Random assignment, control group
Conclusion: Drug likely caused the improvement
Final answer:

Yes, causation can be inferred. The random assignment balances confounding variables across groups, and the control group provides a baseline for comparison. The significant difference in outcomes is likely due to the treatment.

Requirements for Causal Inference:

• Random assignment: Essential for balancing confounding variables

• Control group: Provides baseline for comparison

• Manipulation: Researcher actively applies treatment

• Temporal sequence: Treatment precedes outcome measurement

⚠️ Important Distinction
Causal inference is about cause-and-effect relationships within the study, while generalizability is about applying results to broader populations.
3 Combined Inference
Exercise 3
A researcher uses random sampling to select 1,000 high school students from across the country, then randomly assigns them to two different teaching methods. The study finds that Method A is more effective than Method B. What can be concluded about causation and generalizability?
Definition:

Scope of Inference: The combination of population generalizability and causal inference that determines what conclusions can be drawn from a study.

Step 1: Analyze sampling method

Random sampling from high school students nationwide → Good for generalizability

Step 2: Analyze assignment method

Random assignment to teaching methods → Good for causal inference

Step 3: Determine population inference

Results can be generalized to all high school students nationally

Step 4: Determine causal inference

Method A causes better outcomes than Method B for these students

Random Sampling: National high school students
↓
Random Assignment: Teaching methods
↓
Both Population & Causal Inference Possible
Population Inference: YES (national high school students)
Causal Inference: YES (Method A superior to Method B)
Scope: Full scope of inference achieved
Final answer:

Both population inference and causal inference are possible. The random sampling allows generalization to all national high school students, and the random assignment allows causal inference that Method A is more effective than Method B.

Inference Matrix:
Design Feature Population Inference Causal Inference
Random Sampling + Random Assignment ✓ Possible ✓ Possible
Random Sampling Only ✓ Possible ✗ Not Possible
Random Assignment Only ✗ Limited ✓ Possible
Generalizability and Inference Concepts
\(\text{Scope of Inference} = \text{Population Inference} \cap \text{Causal Inference}\)
Inference Intersection
🎯 Population Inference
Definition: Ability to generalize findings to a larger population
Requires: Random sampling from target population
Concern: External validity (to whom do results apply?)
Cause-and-Effect Inference
Definition: Ability to conclude that X causes Y
Requires: Random assignment to treatment groups
Concern: Internal validity (is the relationship causal?)
🌐 Generalizability
Definition: Extent to which results apply beyond the study sample
Factors: Sampling method, sample representativeness, study context
Limitation: Even valid studies may have limited generalizability
Key Principle: Random sampling enables population inference.
Key Principle: Random assignment enables causal inference.
Key Principle: Both are needed for complete inference scope.
Memory Aid: "Random sampling for populations, random assignment for causation."
Quick Check: Ask: "From whom was the sample drawn?" and "How were participants assigned to groups?"
Solution: Exercises 4 to 5
4 Observational Study Limitations
Exercise 4
A study finds that people who drink green tea regularly have lower rates of heart disease. The study surveyed 10,000 people and found a strong association. Can causation be inferred? What about generalizability? Explain your reasoning.
Definition:

Observational Study: A study where researchers observe subjects without manipulating variables, preventing causal inference due to potential confounding variables.

Step 1: Identify the study type

Observational study (survey) with no manipulation of variables

Step 2: Assess for random assignment

No random assignment - people chose their own tea consumption

Step 3: Evaluate causal inference possibility

No causal inference possible due to confounding variables

Step 4: Assess generalizability

Depends on how the 10,000 people were selected

Observational Study: No Random Assignment
↓
Causal Inference: NOT POSSIBLE
↓
Association Only: Green tea linked to lower heart disease
Causal Inference: NO (observational study)
Population Inference: Depends on sampling method
Conclusion: Association exists, causation uncertain
Final answer:

Causation cannot be inferred because this is an observational study with no random assignment. Generalizability depends on the sampling method used to select the 10,000 people. The study shows association but not causation.

Observational Study Limitations:

• Confounding variables: Other factors may explain the association

• No causal inference: Cannot establish cause-and-effect relationships

• Self-selection bias: Tea drinkers may differ from non-drinkers in many ways

• Alternative explanations: Healthy lifestyle factors may be responsible

5 Study Design Evaluation
Exercise 5
A researcher wants to determine if a new teaching method improves student performance. She selects 200 students from her own school and randomly assigns half to the new method and half to traditional instruction. The new method shows better results. Evaluate the scope of inference for this study.
Definition:

Scope of Inference Evaluation: A systematic assessment of what conclusions can be drawn based on the study's design features and limitations.

Step 1: Identify sampling method
Convenience sampling (students from one school) → Limited population inference
Step 2: Identify assignment method
Random assignment to teaching methods → Good for causal inference
Step 3: Evaluate population inference
Limited to students similar to those at this specific school
Step 4: Evaluate causal inference
Possible due to random assignment
Population Inference: LIMITED (specific school context)
Causal Inference: POSSIBLE (due to random assignment)
Overall Scope: Partial inference - causal but limited generalizability
Final answer:

The study allows causal inference (the new method likely caused the improvement) due to random assignment. However, population inference is limited to students similar to those at this specific school due to convenience sampling.

Design Trade-offs:

• Convenience vs. Rigor: Convenience sampling makes studies easier but limits generalizability

• Internal vs. External Validity: Can have strong internal validity (causal inference) but weak external validity (generalizability)

• Practical Constraints: Real-world limitations often affect optimal design choices

Tip: Always evaluate both population and causal inference separately.
Key Note: A study can have strong causal inference but limited generalizability.
Warning: Never infer causation from observational studies without proper experimental design.
Comprehensive Summary: Generalizability and Scope of Inference
\(\text{Valid Inference} = \text{Random Sampling} \lor \text{Random Assignment}\)
Inference Conditions
📚 Comprehensive Summary: Generalizability and Scope of Inference

🔍 Definitions of Key Concepts

Generalizability: The extent to which findings from a sample can be applied to a larger population beyond the sample studied.
Population Inference: The ability to draw conclusions about a larger population based on sample data, enabled by random sampling.
Causal Inference: The ability to conclude that one variable causes changes in another variable, enabled by random assignment.
Scope of Inference: The combination of population and causal inference that determines the full range of conclusions that can be drawn from a study.
External Validity: The extent to which study results can be generalized to other populations, settings, or times.
Internal Validity: The extent to which a study establishes a trustworthy cause-and-effect relationship.

⚖️ Core Rules and Principles

Rule 1 - Random Sampling: Required for population inference; enables generalization to the target population.
Rule 2 - Random Assignment: Required for causal inference; eliminates confounding variables.
Rule 3 - Independence: Population inference and causal inference are independent; a study can have one without the other.
Rule 4 - Study Design: Experimental studies enable causal inference; observational studies do not.

⚙️ Step-by-Step Methods for Inference Evaluation

  1. Identify the sampling method: Was random sampling used?
  2. Identify the assignment method: Was random assignment used?
  3. Evaluate population inference: Can results be generalized?
  4. Evaluate causal inference: Can causation be concluded?
  5. Determine scope of inference: What conclusions are valid?
  6. Consider limitations: What factors might restrict inference?

🧪 Examples: From Simple to Advanced

Simple Example: Survey of randomly selected town residents about voting preferences. Result: Population inference possible (to town residents), causal inference not applicable.
Intermediate Example: Random assignment of patients to drug vs. placebo. Result: Causal inference possible (drug effectiveness), population inference limited to similar patients.
Advanced Example: Random sampling of high school students nationwide, then random assignment to teaching methods. Result: Both population and causal inference possible.

💡 Tips, Tricks, and Common Pitfalls

Tip 1: Random sampling is for populations, random assignment is for causation.
Tip 2: Observational studies can show association but not causation.
Common Pitfall: Assuming that correlation implies causation in observational studies.
Common Pitfall: Overgeneralizing results from convenience samples to broader populations.
Trick: "SPA" - Sampling for Populations, Assignment for causation.

📝 Key Notes for Memorization

Memory Device: "Random sampling for populations, random assignment for causation" (RS-RA rule).
Key Equation: Valid Inference = Random Sampling OR Random Assignment
Golden Rule: Without random assignment, you cannot infer causation.
Quality Check: Always ask "who" (population) and "what caused what" (causation) when evaluating studies.

📋 Inference Decision Matrix

Study Design Random Sampling Random Assignment Population Inference Causal Inference
Survey (Random) ✓ ✗ ✓ ✗
Experiment (Random) ✗ ✓ ✗ (limited) ✓
Complete RCT ✓ ✓ ✓ ✓
Observational ✗ ✗ ✗ (limited) ✗
Research Hierarchy: Randomized controlled trials with random sampling provide the strongest evidence for both types of inference.

Questions & Answers

Question: I'm confused about when a study can have both population and causal inference. Can you explain the difference?

Answer: These are two different types of inference that require different design features:

  • Population Inference: "To whom do these results apply?" Requires random sampling from the target population
  • Causal Inference: "Does X cause Y?" Requires random assignment to treatment groups

Analogy: Population inference is like asking "does this work for other people?" Causal inference is like asking "did the treatment actually cause the effect?"

Example: A study with random sampling from all high school students AND random assignment to teaching methods can support both types of inference.

Question: Why can't we infer causation from observational studies even if we find a strong association?

Answer: Observational studies have a fundamental problem called "confounding variables":

  • Self-selection: People choose their own behaviors/exposures
  • Confounding: Other variables might explain the association
  • No manipulation: Researchers don't control the variable of interest
  • Alternative explanations: Third variables might cause both the exposure and outcome

Example: People who exercise more might live longer, but maybe healthy people are more likely to exercise AND live longer, not because exercise causes longevity.

Result: We can only say there's an association, not causation.

Question: Can a study have poor population inference but good causal inference? Give an example.

Answer: Yes, absolutely! This is very common in clinical trials:

  • Example: A drug trial recruits patients from a single hospital through convenience sampling, but randomly assigns them to treatment or control groups
  • Causal inference: Excellent - random assignment allows strong causal conclusions
  • Population inference: Poor - results may not apply to patients elsewhere or with different characteristics

Result: Researchers can confidently say "the drug causes improvement" but only for patients similar to those in the study.

Real-world: Many medical studies have strong internal validity (causal inference) but limited external validity (population inference).