Free math worksheets

- Check whether the explanatory variable was assigned by the researchers. - Distinguish observing a relationship from creating comparable treatment groups.

1. The study observes existing behavior rather than assigning exercise amounts. 2. It can show an association between exercise time and resting heart rate, but other variables may help explain the relationship. 3. Therefore, the study by itself does not establish causation.

No. It shows an association, not a cause-and-effect conclusion.

- Identify whether the random step chooses who enters the study or determines a treatment they receive. - Match that random step to the kind of conclusion it can support.

1. Random selection helps make the sample representative of the district's high school student population. 2. No treatment is assigned, so the study does not create the conditions needed for a causal comparison. 3. Therefore, random selection supports generalization, not causation.

It supports generalizing the survey result to high school students in the district, not making a causal claim.

- Compare the observed line with the region where most simulated outcomes appear. - Results deep in a tail occur less often under the model used to generate the simulation.

1. Most simulated differences are concentrated near \(0\). 2. The observed difference is marked far in the right tail, where very few simulated results occur. 3. Therefore, the observed difference is unusual under the no-effect model and provides evidence against that explanation.

Unusual. The observed difference is far in the tail, so the simulation provides evidence against the no-effect model.

- Ask whether anyone was randomly assigned to the proposed cause. - Look for a third factor that changes both variables during the same months. - An observational association can have alternative explanations.

1. The evidence is observational: no one was randomly assigned to buy ice cream, so the association does not isolate a causal effect. 2. Temperature or season is a plausible confounder because warmer months can increase both ice-cream sales and swimming exposure, which can increase drowning incidents.

The study shows an observational association, not causation. Temperature or season is a plausible confounder affecting both ice-cream sales and swimming exposure.

- Use the sampling method to decide the scope of generalization. - Use the assignment method to decide whether causation is supported. - Random sampling and random assignment answer different inference questions.

1. Because the adults were selected with a simple random sample, the observed association can be generalized to the adult population represented by the sampling frame, subject to ordinary sampling and measurement limitations. 2. Exercise was observed rather than randomly assigned, so confounding or reverse causation can explain the association. The design therefore does not establish a causal effect.

The random sample supports a population-level association between exercise and sleep, but not a causal claim because exercise was not randomly assigned.

- Consider how treatment knowledge can influence both participants and evaluators. - Identify exactly which information blinding hides. - No single design feature controls every source of bias.

1. In Trial B, participants' expectations can change reported symptoms, and evaluators' expectations can influence measurement or interaction. 2. Double blinding in Trial A reduces both expectation effects tied to treatment knowledge and evaluator measurement bias. 3. Blinding does not eliminate other problems such as attrition, unrepresentative recruitment, poor adherence, or faulty outcome measures.

Trial A is more credible because double blinding reduces participant-expectation and evaluator-measurement effects. Blinding does not guarantee an unbiased result because attrition, recruitment, adherence, and measurement problems can remain.

- Compare the observed effect size with the decision threshold chosen before the analysis. - Keep evidence that an effect is nonzero separate from evidence that it is large enough to matter. - Sample size affects statistical precision, not the practical threshold itself.

1. The result is not practically significant because the observed reduction of \(0.2\) minute is far below the prespecified \(2\)-minute threshold. 2. With a very large sample, a small effect can be estimated precisely enough to produce strong statistical evidence against a no-effect model even when the effect is too small to matter operationally.

No. The \(0.2\)-minute reduction is below the \(2\)-minute practical threshold. A very large sample can make a tiny effect statistically detectable without making it practically important.

- Compare the requested input with the range of inputs used to fit the model. - A good fit inside one range does not determine behavior far outside it. - Ask what evidence the data actually provide near \(105^\circ\text{F}\).

1. The prediction at \(105^\circ\text{F}\) is a far extrapolation beyond the observed range of \(40^\circ\text{F}\) to \(80^\circ\text{F}\). 2. A relationship that is approximately linear within the fitted range can bend, level off, or change under more extreme conditions, so the data do not establish that the fitted line remains valid at \(105^\circ\text{F}\).

The prediction is an unsupported extrapolation far beyond the data range; a linear pattern from \(40^\circ\text{F}\) to \(80^\circ\text{F}\) need not continue to \(105^\circ\text{F}\).

- Compare the unit represented by each data point with the unit named in the conclusion. - Group averages can behave differently from relationships within groups. - Do not infer an individual relationship merely from an aggregate association.

1. The evidence compares county-level averages, while the conclusion concerns individual people. 2. Group averages do not show that the same relationship holds within counties or for individual residents. County characteristics such as income or school funding can also affect both averages. 3. Therefore, the aggregate association cannot by itself justify the individual-level claim.

The conclusion commits an aggregate-to-individual inference error: a relationship between county averages does not establish the same relationship for individual residents.

- Ask whether the proposed outcome could also influence the proposed cause. - Then look for a third factor connected to both measured variables. - The two explanations should be statistically different mechanisms.

1. Reverse causation is plausible: teenagers experiencing anxiety may turn to screens more often, so anxiety could partly influence screen time. 2. A confounder such as sleep disruption, social isolation, family stress, or school workload could affect both screen use and anxiety. 3. Because the survey is observational and cross-sectional, these alternatives prevent the association from establishing a causal direction.

For example, anxiety could lead to more screen use, and a third variable such as sleep disruption could increase both screen time and anxiety. The survey therefore does not establish that screen time causes anxiety.

- Separate what the model predicts from what management actually wants to measure. - Think of cases where the proxy and the desired outcome move in opposite directions. - Broad constructs usually require several valid indicators.

1. The model predicts email count, but email count is an unvalidated proxy for the broader construct of productivity. 2. High counts can reflect fragmented or unnecessary communication, while valuable focused work may generate few emails. 3. A better evaluation uses several role-appropriate outcomes, such as completed-work quality, timeliness, customer results, and peer or supervisor assessment.

Email count is not a validated measure of productivity: many emails can reflect inefficiency, while productive work may require few messages. Use multiple job-relevant quality and outcome measures instead.

- Judge predictive performance using data that were not used to fit the models. - Compare training performance with test performance for the highly flexible model. - Perfect fit to training observations can include accidental noise.

1. The quadratic model should be preferred because its test error of \(4\) units is much smaller than the degree-9 model's \(18\) units. 2. The degree-9 polynomial fits the training data perfectly but generalizes poorly to new data, which is overfitting.

Prefer the quadratic model. The degree-9 polynomial demonstrates overfitting: perfect training fit but much worse performance on new data.

- Randomization creates a probability mechanism, not guaranteed equality of every group summary. - Consider how much chance imbalance is possible with a small sample. - Separate an unusual realized imbalance from evidence that the assignment process itself was invalid.

1. Random assignment balances groups in expectation over repeated experiments; it does not guarantee identical baseline summaries in every realized sample. 2. With only \(20\) participants, chance imbalance can be substantial. 3. The difference should be reported and considered in analysis, but the imbalance alone does not prove manipulation or failure of the randomization procedure.

Valid random assignment can still produce unequal baseline means by chance, especially in a small sample. The imbalance alone does not prove that randomization failed.

- The test set is separate from training, so focus on a different requirement for useful evaluation. - Compare the conditions represented in the test set with the conditions expected in deployment. - A held-out set can still be biased for the intended use population.

1. Although the April observations are genuinely held out from training, the test set does not represent the full conditions under which the model will be deployed. 2. Clear weekdays may have more regular traffic and fewer unusual delays than weekends or periods of rain or snow. 3. Therefore, the held-out accuracy can be systematically optimistic for the intended deployment population.

The test set is independent of training but unrepresentative of deployment. Restricting evaluation to clear weekdays can make accuracy look better than it will be on weekends or in adverse weather.

- Ask what determined whether each completed study became visible. - Selection can operate on whole studies, not only on participants. - Search for eligible evidence before filtering by its result.

1. The review has publication bias because study visibility depends on whether results are favorable or statistically significant. 2. Omitting null studies makes the evidence appear more consistently positive than the full set of completed studies. 3. A stronger review should seek registrations, unpublished results, completed protocols, and all prespecified outcomes from every eligible study.

The review has publication bias. It should include registered, unpublished, and null studies, along with all prespecified outcomes, rather than selecting studies according to their results.

- Separate a total from a per-person average. - Use the relevant denominator for the word “generosity.” - Several components can change in opposite directions while the total rises.

1. The first-year mean is \(\frac{20000}{100}=\$200\) per donor. 2. The second-year mean is \(\frac{40000}{400}=\$100\) per donor. 3. Total donations doubled, but the average donation per donor fell by \(\$100\), or \(50\%\), so the headline's claim about individual generosity is unsupported. 4. A more accurate summary is that many more donors produced a higher total despite a lower average gift.

a) \(\$200\) and \(\$100\) per donor. b) The headline is misleading; average giving fell by \(50\%\). c) Total donations doubled because the donor count quadrupled, while the mean gift was cut in half.

- Identify whether the operational decision concerns an average or one future observation. - Individual outcomes vary around the mean even if the mean were known precisely. - Match the interval's target to the quantity the manager needs to plan for.

1. The confidence interval \((490,510)\) estimates uncertainty in the mean daily demand, not the variability of an individual future day. 2. The prediction interval \((420,580)\) includes both uncertainty about the mean and natural day-to-day variation. 3. Staffing one specific future day therefore calls for the prediction interval.

Use the prediction interval \((420,580)\), because the decision concerns one future day's demand rather than the population or period mean.

- Use the evaluation measure tied directly to the stated prediction goal. - Compare both methods on the same held-out period. - A goodness-of-fit summary and out-of-sample prediction error do not answer the same question.

1. The seasonal baseline is better on the stated prediction criterion because its mean absolute error is \(8\) units rather than \(12\). 2. Test-period prediction error directly measures performance on the goal being evaluated, while \(R^2\) is a different fit summary. 3. A high \(R^2\) therefore does not compensate for worse out-of-sample prediction error than a simple baseline.

The seasonal baseline predicts better because its test mean absolute error is \(8\) rather than \(12\) units. The model's high \(R^2\) is a different fit measure and does not override worse test error.

- Use the method of selecting participants to judge generalization. - Use the method of assigning conditions to judge causation. - Do not treat random sampling and random assignment as interchangeable.

1. Study A can support a population-wide association because it uses a simple random sample, but it cannot establish causation because sleep time was not assigned. 2. Study B can support a causal comparison for its participants because the schedules were randomly assigned, but volunteers may not represent a larger student population. 3. Random sampling supports generalization, while random assignment supports causal comparison. Each study includes only one of these design features.

a) Study A. b) Study B. c) Random sampling supports generalization; random assignment supports causation. Neither study has both features.

- Look for a systematic shape in the residuals rather than expecting every point to lie on zero. - Compare the display with an arch or other repeated curved pattern. - Random-looking scatter above and below zero does not by itself indicate missed curvature.

1. The residuals are scattered above and below zero without a clear arch, curve, or systematic trend as \(x\) increases. 2. Therefore, this residual plot does not show clear evidence of missing curvature and is consistent with a linear relationship.

No. The residuals show no clear systematic curved pattern around zero, so this plot is consistent with a linear model.

- The observed statistic must be known before simulated results can be compared with it. - One shuffled assignment produces one value in the no-effect distribution. - The final proportion uses extreme simulated trials as the numerator.

1. First calculate the observed difference, so C comes first. 2. Shuffle the treatment labels and calculate one simulated difference, so B comes next. 3. Repeat that process many times, so D follows. 4. Finally count the simulated differences at least as extreme as the observed difference, so A is last. 5. The tail proportion is the number of simulated differences at least as extreme as the observed difference divided by the total number of shuffled trials.

a) C, B, D, A. b) Divide the number of simulated results at least as extreme as the observed result by the total number of randomization trials.

- Treat random assignment and random sampling as serving different inferential purposes. - Use the tail proportion to judge compatibility with a no-effect model. - Match the scope of the conclusion to how participants entered the study.

1. Random assignment makes the treatment groups comparable on average, so the small tail proportion supports a causal conclusion about the app for the participating students in this experiment. 2. Because participants volunteered rather than being randomly sampled, the result cannot automatically be generalized to all students at the school. 3. Under a no-effect randomization model, only about \(1.0\%\) of simulated assignments produced a difference at least as favorable as \(3.2\) points.

a) The study supports a causal effect of the app for the participating students in this experiment. b) It does not justify generalizing the effect to every student at the school. c) A difference at least this large occurred in about \(1\%\) of no-effect simulations.

- It is easier to compute the complement event in which none of the tests produces a false flag. - Then compare the resulting family-level probability with the single-test rate. - The analysts had twenty opportunities to find an apparently unusual result.

1. The probability of no false flags is \((0.95)^{20}\approx0.3585\). 2. Therefore, the probability of at least one false flag is \(1-0.3585\approx0.6415\). 3. With about a \(64\%\) chance of at least one false flag under the no-effect model, selecting the one favorable result after twenty searches provides much weaker evidence than a single prespecified test would.

\(1-(0.95)^{20}\approx0.6415\). Because at least one false flag is common across twenty independent tests, the selected tail probability \(0.030\) is weak evidence unless the multiple search is accounted for.

- Look at the signs and sizes in sequence rather than averaging them. - A useful residual plot should not leave a visible structure unexplained. - Match the shape of the residual pattern to a missing feature in the model.

1. The residuals begin negative, become positive in the middle, and return to negative, forming a systematic curved pattern. 2. A linear model is not appropriate because residuals should fluctuate without a clear pattern around zero; the model underpredicts in the middle and overpredicts at both ends. 3. A curved relationship, such as a quadratic model, should be considered and checked against the data.

a) The residuals form an arch-shaped pattern. b) No. Their systematic pattern shows that the line misses curvature. c) Consider a curved model, such as a quadratic, and recheck residuals.

- Use the axis values, not the apparent bar-height ratio. - Calculate both an absolute percentage-point difference and a relative change. - Compare the visible bar heights with the actual percentages they represent.

1. Option A has \(48\%\) support and Option B has \(52\%\). 2. The difference is \(52-48=4\) percentage points. The relative increase is \(\frac{52-48}{48}\approx0.0833=8.33\%\). 3. The y-axis begins at \(47\%\) rather than \(0\%\), so the visible bar heights are \(48-47=1\) and \(52-47=5\). The second visible height is five times the first, but that comparison uses distances above an arbitrary truncated baseline, not the support percentages.

a) A: \(48\%\); B: \(52\%\). b) Difference: \(4\) percentage points; relative increase: about \(8.33\%\). c) The visible heights are \(1\) and \(5\) percentage points above a truncated \(47\%\) baseline. The fivefold visual comparison is not a comparison of the actual support percentages.

- Notice that selection used an extreme first measurement. - Ask what may happen on a second measurement without any treatment. - A comparison group should experience the same timing and selection process.

1. The players were selected for extreme low results that partly reflect temporary random variation. 2. On a later week, the unusually poor variation is unlikely to repeat as strongly, so percentages can move closer to typical levels even if the routine has no effect. 3. A stronger design randomly assigns similarly low-performing players to the new routine or a comparison routine and compares later changes.

The selected extreme lows can be followed by less extreme results simply because temporary poor variation does not repeat; this is regression toward the mean. Randomly assign comparable low-performing players to treatment and control routines to test the effect.

- Count correct predictions separately for each actual class. - Examine performance on the rare outcome rather than only the total. - A useful metric should reflect the kind of error that matters for the task.

1. The model correctly labels all \(950\) legitimate transactions, so accuracy is \(\frac{950}{1000}=95\%\). 2. It detects \(0\) of \(50\) fraudulent transactions, so its fraud-detection rate is \(0\%\). 3. The classes are highly imbalanced, so a model can obtain high overall accuracy by always predicting the common class while failing completely on the important rare class. 4. The report should include the fraud-detection rate, false-alarm rate, or a full confusion matrix.

a) \(95\%\). b) \(0\%\). c) The common legitimate class dominates the accuracy calculation and hides total failure on fraud. d) Report fraud-detection rate and false-alarm rate, or the full confusion matrix.

- Use actual positives as the denominator for detection and actual negatives for false alarms. - Compare both rates before judging a threshold. - A threshold decision reflects the costs of different mistakes, not only mathematical accuracy.

1. At the low threshold, the detection rate is \(\frac{90}{100}=90\%\), and the false-alarm rate is \(\frac{180}{900}=20\%\). 2. At the high threshold, the detection rate is \(\frac{65}{100}=65\%\), and the false-alarm rate is \(\frac{45}{900}=5\%\). 3. Raising the threshold reduces false alarms but misses more positive cases. 4. The best threshold depends on the relative costs of missed positives, false alarms, follow-up resources, and the intended use.

a) Low: \(90\%\) detection and \(20\%\) false alarms. High: \(65\%\) detection and \(5\%\) false alarms. b) The higher threshold lowers both detections and false alarms. c) The choice depends on the consequences and costs of the two error types.

- Combine group totals rather than averaging the two group means equally. - Weight each mean by its group size. - Ask whether response status is associated with the measured variable.

1. The combined salary total is \(800\cdot62{,}000+200\cdot120{,}000=73{,}600{,}000\) dollars. 2. The full-sample mean is \(\frac{73600000}{1000}=\$73{,}600\). 3. The respondent-only mean is \(73{,}600-62{,}000=\$11{,}600\) too low. 4. Nonresponse is related to salary, so treating the observed salaries as representative of all selected employees is inappropriate.

a) \(\$73{,}600\). b) The respondent-only mean underestimates by \(\$11{,}600\). c) The missingness is related to the outcome, producing nonresponse bias.

- Count how many subgroup opportunities were examined before one was highlighted. - Ask whether the subgroup was chosen before or after seeing the outcomes. - Post hoc findings require stronger confirmation than prespecified tests.

1. The subgroup was selected after the analysts inspected many possible subgroup results, creating multiple opportunities for a chance finding. 2. A tail probability of \(0.04\) is therefore less persuasive than it would be for one prespecified subgroup analysis. 3. The result is better treated as a hypothesis to be tested in a planned, independent replication.

Because the subgroup was found after searching \(12\) groups, the result may be a chance selection from multiple comparisons. It should be treated as exploratory and confirmed with a prespecified independent analysis.

- Compare both centers and spreads shown by the boxes. - Consistency is reflected by how tightly outcomes cluster. - One summary statistic cannot determine an entire distribution.

1. Service A has median \(30\) and interquartile range \(32-28=4\) minutes. Service B has median \(25\) and interquartile range \(45-15=30\) minutes. 2. Service A is more consistent because its central half and full whisker range are much narrower. 3. Equal means describe only one measure of center; Service B's broad, skewed distribution can have the same mean while producing far less predictable delivery times.

a) A: median \(30\), IQR \(4\). B: median \(25\), IQR \(30\). b) Service A. c) Reliability depends on variability and distribution shape, not only the mean.

- First recover correctly classified negatives from the false-positive counts. - Use actual positives as the denominator for missed-positive rate. - A shared overall percentage can be built from very different kinds of errors.

1. Group A has \(90\) correct positives and \(90\) correct negatives, so accuracy is \(\frac{180}{200}=90\%\). 2. Group B has \(10\) correct positives and \(170\) correct negatives, so accuracy is also \(\frac{180}{200}=90\%\). 3. Group A misses \(10\) of \(100\) positives, a \(10\%\) missed-positive rate. Group B misses \(10\) of \(20\), a \(50\%\) rate. 4. Different class proportions allow the same total accuracy to hide a much worse error rate on positive cases in Group B.

a) Both groups have \(90\%\) accuracy. b) Group A: \(10\%\); Group B: \(50\%\). c) Accuracy combines error types and is affected by group class proportions, so it can conceal unequal missed-positive rates.

- Weight each group effect by its share of the population. - An average can combine effects with opposite signs. - Decision-making may require knowing who benefits and who may be harmed.

1. The weighted average effect is \(0.80\cdot10+0.20\cdot(-5)=8-1=7\) units. 2. The positive average hides that the treatment may be harmful for Group 2, so the same conclusion does not apply uniformly to every subgroup. 3. The report should include group-specific effect estimates, uncertainty, group sizes, and evidence about whether the difference between subgroup effects is reliable.

a) \(+7\) units. b) The average conceals a negative estimated effect in Group 2. c) Report subgroup effects with uncertainty and evaluate the evidence for effect differences.

- Ask when each predictor becomes available relative to the prediction decision. - A held-out set is useful only when it contains the same information that would be available in real use. - Random splitting does not remove a predictor that already reveals future information.

1. “Course completed” is recorded only after the final outcome is known, so it leaks future or outcome information into the predictor set. 2. A random train-test split does not solve the problem because the leaked variable appears in both sets. 3. In real use before the exam, that information would be unavailable, so the reported test setting does not reproduce the intended prediction task.

The model uses future-information leakage. Because the leaked predictor appears in both random splits, the \(99\%\) test accuracy measures an unrealistic task rather than true pre-exam prediction performance.

- Compare the population on which performance was validated with the population now being scored. - Look for changes in predictor distributions and operating conditions over time. - A performance estimate is tied to the distribution on which it was evaluated.

1. The applicant distribution and possibly the relationship between predictors and default outcomes have shifted over time. 2. The \(2018\) validation data therefore no longer represent the \(2026\) deployment population. 3. This distribution shift explains why the old accuracy estimate cannot be assumed to describe current performance.

The evidence suggests distribution shift. The \(2018\) validation population no longer matches \(2026\) applicants, so the old performance estimate is not reliable for current deployment.

- Calculate change once relative to the original risk and once directly on the percentage scale. - Translate both rates into counts for a common population size. - A relative statement can sound large when the baseline rate is small.

1. The relative reduction is \(\frac{2\%-1\%}{2\%}=\frac{1}{2}=50\%\). 2. The absolute reduction is \(2\%-1\%=1\) percentage point. 3. Without treatment, the expected count is \(10{,}000\cdot0.02=200\); with treatment, it is \(10{,}000\cdot0.01=100\), a difference of \(100\) events. 4. The relative claim is mathematically correct but incomplete without the starting risk and absolute change.

a) \(50\%\) relative reduction. b) \(1\) percentage point absolute reduction. c) About \(100\) fewer events per \(10{,}000\) people. d) The claim is correct but should also report the baseline risk and absolute reduction.

- Calculate each change relative to its own starting value. - Compare periods that differ in whether an unusual disruption occurred. - A claim can be numerically correct yet misleading because of endpoint selection.

1. The February-to-March increase is \(\frac{840-600}{600}=0.40=40\%\). 2. The January-to-March change is \(\frac{840-1000}{1000}=-0.16=-16\%\). 3. The year-over-year March change is \(\frac{840-830}{830}\approx0.0120=1.20\%\). 4. The advertised comparison starts from an unusually low outage month and therefore exaggerates ordinary growth; broader and comparable periods give a different picture.

a) \(40\%\). b) January to March: \(-16\%\); year-over-year March: about \(+1.20\%\). c) The selected outage month creates a misleadingly low baseline and overstates growth.

- Compare the center and spread of residuals separately. - Prediction uncertainty should reflect the local size of typical errors. - A single error margin assumes similar variability everywhere.

1. The residuals remain centered near zero, but their spread increases as \(x\) increases. Constant residual variability is therefore questionable. 2. Predictions should be most precise at low \(x\), less precise in the middle, and least precise at high \(x\). 3. A constant \(\pm3\) rule is wider than needed for many low-\(x\) cases but far too narrow for high-\(x\) cases, so it misrepresents changing uncertainty.

a) Residual variability increases with \(x\), so constant spread is questionable. b) Prediction intervals should widen as \(x\) increases. c) One fixed margin ignores the strong change in residual spread across the domain.

- Compare the effect across levels of the environmental condition. - A single effect is questionable when the change depends strongly on another condition. - Context-specific estimates can be more informative than an unweighted average.

1. The report treats a context-dependent effect as one constant effect and therefore misrepresents both dry and wet conditions. 2. The fertilizer effect changes with moisture conditions rather than remaining constant. 3. A more accurate conclusion is that the estimated increase is about \(5\) bushels per acre when dry and \(20\) when wet; any overall average must be weighted by the frequency of those conditions and should not replace the separate effects.

a) The average is incorrectly applied uniformly to two conditions with very different effects. b) The fertilizer effect depends on whether conditions are dry or wet. c) Report the separate \(5\)- and \(20\)-bushel effects and, if needed, a properly weighted overall average.

- Track whether information from the supposed test set changed any modeling decision. - Selecting the largest value among many noisy estimates favors positive random fluctuation. - An independent test set must remain untouched until model choices are finished.

1. Because test-set results influenced which model was selected, the test set became part of the model-development process rather than remaining an independent final evaluation. 2. Choosing the maximum among many noisy test accuracies favors models that benefited from positive random fluctuation on that particular test set. 3. The selected accuracy is therefore likely biased upward relative to performance on genuinely new data.

Repeated test-set reuse makes the test data part of model selection. Choosing the best of \(50\) noisy test results favors an unusually high value, so the reported accuracy is likely optimistic.

- Use extreme simulated outcomes divided by all simulated outcomes. - Keep the conditioning direction clear: the model is assumed when the probability is calculated. - Distinguish probability of data under a model from probability of a model given data.

1. The estimated tail probability is \(\frac{12}{1000}=0.012\). 2. If the no-effect model and randomization process were appropriate, about \(1.2\%\) of assignments would produce a mean difference at least as large as \(8.0\). 3. The tail probability is calculated assuming the no-effect model; it is not the probability that the model itself is true.

a) \(0.012\). b) Under the no-effect model, a result at least this large occurs in about \(1.2\%\) of randomizations. c) No. It is a probability of simulated data under the model, not a probability assigned to the model.

- Compute rates using applicants within the same department as denominators. - Then recompute after pooling each gender across departments. - Compare how each group is weighted toward the high-rate and low-rate departments.

1. In Department A, the rates are \(\frac{80}{100}=80\%\) for women and \(\frac{18}{20}=90\%\) for men. In Department B, they are \(\frac{2}{20}=10\%\) for women and \(\frac{60}{100}=60\%\) for men. 2. Overall, women have \(\frac{82}{120}\approx68.3\%\), while men have \(\frac{78}{120}=65\%\). 3. More women applied to high-admission Department A, while more men applied to low-admission Department B. Different group weights reverse the aggregated comparison.

a) Men have higher admission rates in both departments: \(90\%\) versus \(80\%\) in A and \(60\%\) versus \(10\%\) in B. b) Overall, women have about \(68.3\%\) admitted and men \(65\%\). c) The applicant groups are distributed very differently across departments with very different base admission rates.

- Random assignment balances groups before later events occur. - Ask who is missing from each final comparison and why. - A trustworthy report should show how different reasonable assumptions about missing outcomes affect the conclusion.

1. Differential dropout can destroy the comparability created by random assignment among the participants who remain. 2. If exercise participants with little improvement or difficulty following the program are more likely to leave, the completers can make the program look more effective than it was for everyone assigned. 3. The report should compare dropout reasons and baseline traits, include outcomes for all assigned participants when possible, and show how conclusions change under reasonable assumptions about the missing outcomes.

a) Unequal attrition can make the remaining groups systematically different. b) For example, exercise participants who do not improve may be more likely to drop out. c) Analyze all assigned participants when possible and report how the conclusion changes under reasonable assumptions about missing outcomes.

- Separate goodness of fit from evidence about cause. - Ask what third variable changes systematically with both annual series. - Account for the shared time trend before interpreting the relationship.

1. \(R^2\) describes the strength of a linear association; it does not establish a causal mechanism or rule out confounding. 2. Both variables have strong upward time trends, so later years are high in both series even if the variables are unrelated. 3. A better analysis would account for time trend and inflation, compare year-to-year changes after adjusting for inflation and use predictors connected to a plausible textbook-price mechanism.

The high \(R^2\) can be caused by a shared upward time trend and does not establish causation. A more meaningful analysis should account for the time trend and inflation, then evaluate variables tied to a plausible pricing mechanism.

- An estimate from one sample is not expected to match another exactly. - Compare both the centers and the full plausible ranges. - Investigate methodological differences before making claims about conduct.

1. Different random samples and study conditions can produce different estimates; disagreement does not by itself establish misconduct. 2. Study 1 supports positive effects from \(2\) to \(8\), while Study 2 includes negative, zero, and small positive effects from \(-4\) to \(2\). They share the boundary value \(2\) but give substantially different centers. 3. Researchers should compare protocols, populations, measurements, and attrition, and combine or repeat well-designed studies when appropriate.

a) Sampling variation or design differences can explain disagreement; the intervals alone do not show fraud. b) The first supports a positive effect, while the replication supports values from moderately negative through small positive. c) Audit design differences and conduct or combine additional carefully designed replications.

- Compare the predicted percentage with the observed relative frequency. - State both the direction and size of the discrepancy. - Probability accuracy and ranking performance are different model properties.

1. The observed event rate is \(\frac{90}{200}=0.45=45\%\). 2. The observed rate is \(45\%-30\%=15\) percentage points higher than predicted, so the model underpredicts risk for this group and is poorly calibrated here. 3. Calibration compares predicted probabilities with observed frequencies; ranking ability requires comparing whether cases with higher assigned risks actually tend to experience more events than cases with lower assigned risks.

a) \(45\%\). b) The model underpredicts by \(15\) percentage points for this group. c) Calibration within one risk group does not measure how well the model orders cases across different risk levels.

- Use falsely stopped simulations divided by all no-effect simulations. - Compare both the absolute and multiplicative increase over the target rate. - A decision rule includes when the data are examined and when collection stops.

1. The simulated false-positive rate is \(\frac{142}{1000}=0.142=14.2\%\). 2. This is \(14.2\%-5.0\%=9.2\) percentage points above the intended rate and is about \(\frac{14.2}{5}=2.84\) times as large. 3. Each additional look creates another opportunity for random fluctuation to cross the threshold; the original tail-probability interpretation assumes a fixed analysis plan.

a) \(14.2\%\). b) It is \(9.2\) percentage points higher, about \(2.84\) times the intended rate. c) Repeated optional checking adds chances to stop on a random extreme result.

- Compute each location's change using the same subtraction order. - Subtract the comparison change from the treated change. - Use the untreated location to estimate what might have happened anyway.

1. The treated city changes by \(70-100=-30\) crashes. The comparison city changes by \(88-110=-22\) crashes. 2. The adjusted difference in changes is \((-30)-(-22)=-8\) crashes. 3. A broad trend reduced crashes even without the law, as shown by the comparison city. The raw decline attributes that shared trend to the law, while the adjusted comparison suggests an additional decline of about \(8\) crashes.

a) Treated city: \(-30\); comparison city: \(-22\). b) \(-8\) crashes. c) Most of the decline may reflect a trend affecting both cities rather than the law alone.

- Identify which observations share the same source. - Repeated measurements do not add as much new information as measurements from new independent units. - Either change the unit of analysis or use a model that represents the grouping.

1. Observations from the same student are likely correlated, violating the assumption that all \(300\) observations are independent. 2. Treating repeated observations as independent overstates the amount of separate information and usually makes standard errors and intervals too small. 3. The analysis can summarize each student first or use a method that treats observations from the same student as related; the design can also sample more students rather than only more days per student.

a) Independence is violated within each student's repeated measurements. b) Uncertainty is understated because the effective sample size is less than \(300\). c) Analyze student-level summaries or use a method that accounts for repeated observations from each student, and increase the number of students when possible.

- Consider what happens when many new values are placed exactly at the center. - A filled-in value is not the same as an observed value. - Valid uncertainty should include uncertainty about what was missing.

1. Replacing missing values by the observed mean leaves the completed-data mean at \(75\) if the observed mean was \(75\). 2. Every imputed value lies exactly at the center, so the completed data have artificially reduced variability. 3. The method treats guessed values as known and ignores uncertainty about missing outcomes, causing model uncertainty and confidence-interval width to be understated.

a) It preserves the observed mean of \(75\). b) It artificially reduces variability by adding many central values. c) The analysis ignores uncertainty in the imputed values and therefore appears too precise.

- Keep the groups defined by the random mechanism. - A choice made after assignment can be related to the outcome. - Distinguish the effect of being offered a program from the effect among self-selected users.

1. The assignment-based effect estimate is \(74-70=4\) units. 2. People who choose to participate may differ in motivation, availability, or baseline condition from nonparticipants and controls, so the \(78-70=8\) comparison mixes treatment with self-selection. 3. Comparing all people according to original assignment preserves randomized group comparability and estimates the effect of offering or assigning the program.

a) \(4\) units. b) Actual participation is self-selected after assignment, so the \(8\)-unit comparison is confounded. c) Compare all \(200\) people by their original randomized assignment.

- Begin by splitting the population according to the base rate. - Apply each conditional positive rate within the appropriate group. - Check which event appears after the conditioning bar in each probability.

1. Of \(10{,}000\) people, \(200\) have the condition and \(9800\) do not. 2. The expected true positives are \(200\cdot0.90=180\). The expected false positives are \(9800\cdot0.05=490\). 3. The percentage with the condition among all positive tests is \(\frac{180}{180+490}\approx0.2687=26.87\%\). 4. Sensitivity conditions on actually having the condition, while the requested probability conditions on receiving a positive test; the low base rate and false positives matter.

a) \(180\) true positives and \(490\) false positives. b) About \(26.87\%\). c) Sensitivity is \(P(+\mid C)\), not \(P(C\mid +)\); the population base rate changes the latter probability.

- Ask whether one participant's assigned treatment can change another participant's experience. - Consider what happens when the control group receives part of the active program. - Choose assignment units that reduce contact across treatment conditions.

1. The study's assumption that one group's treatment does not affect outcomes in the other group is threatened by spillover or interference. 2. Control students may receive part of the tutoring benefit, making the groups more similar and often reducing the estimated difference; other social effects could also complicate direction. 3. Researchers could assign more widely separated schools or groups, measure how much sharing occurs, and report how the results change when cross-group exposure is considered.

a) Spillover makes the treatment and control conditions overlap. b) Shared materials can contaminate the control condition and usually shrink the observed contrast. c) Assign more separated groups and measure or account for cross-group exposure.

- Separate how treatment was assigned from how outcomes are later analyzed. - Matching is useful when partners share variation that can cancel in a difference. - Use the same observational unit in the analysis that the randomization created.

1. The original within-pair random assignment still occurred, so discarding labels does not erase the random treatment mechanism or necessarily change the treatment-group mean difference. 2. Matching made partners similar at baseline, and the within-pair differences vary by only \(4\) points compared with \(15\)-point variation among individual outcomes. Ignoring pairs fails to remove that shared baseline variation and therefore loses precision. 3. The analysis should use the \(50\) within-pair differences, centered at the observed mean difference of \(3.2\) points, with a method designed for paired data.

a) No. Treatment was still randomly assigned within pairs, although the analysis ignores useful design information. b) Ignoring matching leaves large between-participant variation in the error instead of canceling it within pairs. c) Analyze the \(50\) A-minus-B pair differences, whose mean is \(3.2\) points.

- Separate uncertainty inside one chosen model from uncertainty about choosing the model itself. - Good historical fit does not make two different future assumptions equivalent. - Show how the conclusion changes under other plausible model assumptions.

1. The report omits model uncertainty: different plausible model structures and assumptions produce very different forecasts. 2. A narrow interval conditional on Model A can describe random error within that model while ignoring the larger uncertainty about which model is appropriate. 3. A responsible report should show forecasts from both justified models, explain their assumptions, evaluate them on relevant validation data, and show how the forecast changes under the different plausible models.

a) Uncertainty about which model and assumptions are appropriate. b) It can look precise while ignoring another plausible forecast far from \(120\). c) Report alternative models, assumptions, validation performance, and sensitivity of conclusions to model choice.

- Evaluate random assignment, sampling, missing outcomes, and uncertainty as separate parts of the design. - Ask whether the set of completers can still be treated as if it arose solely from the original random assignment. - Distinguish a sample average from a population effect and from every individual's response. - A defensible conclusion should preserve both the observed completer difference and the study's design limitations.

1. Random assignment initially creates comparable treatment groups and supports causal inference if outcomes are retained. The much larger dropout in the drink group can destroy that comparability among completers, so the completer-only difference is not a clean randomized estimate of the causal effect. 2. Volunteers from one fitness center are not a random sample of all adults, so broad population generalization is unsupported. 3. The \(4.0\) value is a sample mean difference, not an exact population effect; the confidence interval shows uncertainty about the completer mean difference. Also, an average difference does not imply that every individual changes by the same amount. 4. Shuffling treatment labels only among the observed completers does not reproduce the original random-assignment mechanism if treatment assignment affected who remained in the study. Therefore, the \(0.020\) tail proportion cannot by itself be treated as a clean randomization-test probability for the original experiment. The confidence interval likewise describes uncertainty for the analyzed completers and does not repair attrition bias. 5. A supported conclusion is that drink-group completers had a higher mean heart rate than placebo completers, with an estimated difference of \(4.0\) beats per minute and completer interval \((0.5,7.5)\), but differential attrition weakens causal interpretation and the volunteer sample does not support generalization to all adults.

a) Random assignment initially supports causation, but differential attrition means the completer-only comparison is not a clean causal estimate. b) The volunteer sample from one fitness center does not support generalization to all adults. c) \(4.0\) is an estimate rather than an exact population effect, and a mean effect is not an identical effect for every individual. d) Because dropout differs after assignment, shuffling labels only among completers need not represent the original randomization; the \(0.020\) tail proportion and completer confidence interval do not remove attrition bias. e) Drink-group completers had a higher mean heart rate, but differential attrition weakens causal interpretation and generalization to all adults is unsupported.