Free math worksheets

- Use the sampling method to decide the population scope. - Use the treatment-assignment method to decide whether causation is supported. - Look for a third variable connected to both measured quantities.

1. The random sample can support generalizing an association between exercise and sleep to the adult population represented by the sampling frame. 2. Exercise was observed rather than randomly assigned, so differences may be caused by other variables or reverse direction of influence. 3. Work schedule, health status, age, or stress could affect both exercise time and sleep duration.

a) The study can support a population-level association between exercise and sleep. b) It cannot establish causation because there was no random assignment. c) For example, work schedule or overall health may affect both variables.

- Consider how treatment knowledge can influence both participants and evaluators. - Identify exactly which information blinding hides. - No single design feature controls every source of bias.

1. In Trial B, participants' expectations can change reported symptoms, and evaluators' expectations can influence measurement or interaction. 2. Double blinding in Trial A reduces both expectation effects tied to treatment knowledge and evaluator measurement bias. 3. Blinding does not eliminate other problems such as attrition, unrepresentative recruitment, poor adherence, or faulty outcome measures.

Trial A is more credible because double blinding reduces participant-expectation and evaluator-measurement effects. Blinding does not guarantee an unbiased result because attrition, recruitment, adherence, and measurement problems can remain.

- Compare the prediction input with the range used to fit the model. - A strong fit within one domain does not determine behavior outside it. - Credible prediction requires evidence near the intended input values.

1. The prediction is an extrapolation far outside the observed temperature range. 2. The relationship can bend, level off, change slope, or be affected by different operating conditions beyond the data range even if it is nearly linear within the observed range. 3. Data from temperatures near \(105^\circ\text{F}\) or a validated model with justified behavior over that expanded domain would make the prediction more credible.

a) It is a far extrapolation beyond the model's data domain. b) A local linear pattern need not continue under more extreme conditions. c) Collect relevant high-temperature data or validate a model over the wider range.

- Separate what the model predicts from what management actually wants to measure. - Think of cases where the proxy and the desired outcome move in opposite directions. - Broad constructs usually require several valid indicators.

1. The model predicts email count, but email count is an unvalidated proxy for the broader construct of productivity. 2. High counts can reflect fragmented or unnecessary communication, while valuable focused work may generate few emails. 3. A better evaluation uses several role-appropriate outcomes, such as completed-work quality, timeliness, customer results, and peer or supervisor assessment.

Email count is not a validated measure of productivity: many emails can reflect inefficiency, while productive work may require few messages. Use multiple job-relevant quality and outcome measures instead.

- Ask what determined whether each completed study became visible. - Selection can operate on whole studies, not only on participants. - Search for eligible evidence before filtering by its result.

1. The review has publication bias because study visibility depends on whether results are favorable or statistically significant. 2. Omitting null studies makes the evidence appear more consistently positive than the full set of completed studies. 3. A stronger review should seek registrations, unpublished results, completed protocols, and all prespecified outcomes from every eligible study.

The review has publication bias. It should include registered, unpublished, and null studies, along with all prespecified outcomes, rather than selecting studies according to their results.

- Separate a total from a per-person average. - Use the relevant denominator for the word “generosity.” - Several components can change in opposite directions while the total rises.

1. The first-year mean is \(\frac{20000}{100}=\$200\) per donor. 2. The second-year mean is \(\frac{40000}{400}=\$100\) per donor. 3. Total donations doubled, but the average donation per donor fell by \(\$100\), or \(50\%\), so the headline's claim about individual generosity is unsupported. 4. A more accurate summary is that many more donors produced a higher total despite a lower average gift.

a) \(\$200\) and \(\$100\) per donor. b) The headline is misleading; average giving fell by \(50\%\). c) Total donations doubled because the donor count quadrupled, while the mean gift was cut in half.

- Use the method of selecting participants to judge generalization. - Use the method of assigning conditions to judge causation. - Do not treat random sampling and random assignment as interchangeable.

1. Study A can support a population-wide association because it uses a simple random sample, but it cannot establish causation because sleep time was not assigned. 2. Study B can support a causal comparison for its participants because the schedules were randomly assigned, but volunteers may not represent a larger student population. 3. Random sampling supports generalization, while random assignment supports causal comparison. Each study includes only one of these design features.

a) Study A. b) Study B. c) Random sampling supports generalization; random assignment supports causation. Neither study has both features.

- Look for a systematic shape rather than expecting every residual to equal zero. - Compare this pattern with an arch, trend, or funnel shape. - A satisfactory residual shape is one model check, not a complete guarantee.

1. The residuals are scattered above and below zero without a clear curved or steadily increasing pattern. 2. No. This plot does not show clear evidence of missing curvature, so the residual pattern is consistent with a linear model. 3. Other checks include unusual points, changing residual spread, whether predictions stay within the observed x-range, and whether the data-collection design supports the intended use.

a) The residuals show no clear systematic pattern around zero. b) No; the plot is consistent with a linear relationship. c) For example, check for unusual points, changing spread, or extrapolation beyond the observed range.

- The observed statistic must be known before simulated results can be compared with it. - One shuffled assignment produces one value in the no-effect distribution. - The final proportion uses extreme simulated trials as the numerator.

1. First calculate the observed difference, so C comes first. 2. Shuffle the treatment labels and calculate one simulated difference, so B comes next. 3. Repeat that process many times, so D follows. 4. Finally count the simulated differences at least as extreme as the observed difference, so A is last. 5. The tail proportion is the number of simulated differences at least as extreme as the observed difference divided by the total number of shuffled trials.

a) C, B, D, A. b) Divide the number of simulated results at least as extreme as the observed result by the total number of randomization trials.

- Ask whether researchers assigned anyone to buy ice cream. - Look for a third factor that changes both variables in the same months. - A causal claim requires ruling out alternative explanations for the association.

1. The evidence is observational because no treatment was randomly assigned. 2. Temperature or season is a plausible confounding variable because warm weather can increase both ice-cream sales and swimming exposure. 3. The observed association can be explained by the confounder, so it does not isolate an effect of buying ice cream on drowning risk. 4. A safer design could analyze individual-level observational data while adjusting or comparing within temperature, season, location, and swimming-exposure groups.

a) Observational. b) Temperature or season. c) A common cause can produce both higher ice-cream sales and more swimming-related exposure, so causation is not established. d) Use a carefully controlled observational analysis that accounts for weather and swimming exposure.

- Treat random assignment and random sampling as serving different inferential purposes. - Use the tail proportion to judge compatibility with a no-effect model. - Match the scope of the conclusion to how participants entered the study.

1. Random assignment makes the treatment groups comparable on average, so the small tail proportion supports a causal conclusion about the app for the participating students in this experiment. 2. Because participants volunteered rather than being randomly sampled, the result cannot automatically be generalized to all students at the school. 3. Under a no-effect randomization model, only about \(1.0\%\) of simulated assignments produced a difference at least as favorable as \(3.2\) points.

a) The study supports a causal effect of the app for the participating students in this experiment. b) It does not justify generalizing the effect to every student at the school. c) A difference at least this large occurred in about \(1\%\) of no-effect simulations.

- Separate evidence that an effect is nonzero from evidence that it is large enough to matter. - Use the threshold chosen before seeing the data. - A complete report should describe magnitude as well as uncertainty.

1. The very large sample makes the estimate highly precise, so even a small difference can be unlikely under a no-effect model. 2. The reduction of \(0.2\) minute is below the prespecified \(2\)-minute practical threshold, so it is not practically significant for the decision. 3. The report should include both the effect size, \(0.2\) minute, and its uncertainty or simulation-based significance measure.

a) A huge sample can make a very small effect distinguishable from zero. b) No; \(0.2\) minute is below the \(2\)-minute practical threshold. c) Report the effect size and a measure of its uncertainty or statistical evidence.

- Look at the signs and sizes in sequence rather than averaging them. - A useful residual plot should not leave a visible structure unexplained. - Match the shape of the residual pattern to a missing feature in the model.

1. The residuals begin negative, become positive in the middle, and return to negative, forming a systematic curved pattern. 2. A linear model is not appropriate because residuals should fluctuate without a clear pattern around zero; the model underpredicts in the middle and overpredicts at both ends. 3. A curved relationship, such as a quadratic model, should be considered and checked against the data.

a) The residuals form an arch-shaped pattern. b) No. Their systematic pattern shows that the line misses curvature. c) Consider a curved model, such as a quadratic, and recheck residuals.

- Use the axis values, not the apparent bar-height ratio. - Calculate both an absolute percentage-point difference and a relative change. - Compare the visible bar heights with the actual percentages they represent.

1. Option A has \(48\%\) support and Option B has \(52\%\). 2. The difference is \(52-48=4\) percentage points. The relative increase is \(\frac{52-48}{48}\approx0.0833=8.33\%\). 3. The y-axis begins at \(47\%\) rather than \(0\%\), so the visible bar heights are \(48-47=1\) and \(52-47=5\). The second visible height is five times the first, but that comparison uses distances above an arbitrary truncated baseline, not the support percentages.

a) A: \(48\%\); B: \(52\%\). b) Difference: \(4\) percentage points; relative increase: about \(8.33\%\). c) The visible heights are \(1\) and \(5\) percentage points above a truncated \(47\%\) baseline. The fivefold visual comparison is not a comparison of the actual support percentages.

- Check whether the evidence and the conclusion refer to the same unit: counties or individuals. - Group averages can conceal very different within-group relationships. - The desired claim requires measurements connected at the individual level.

1. The conclusion incorrectly transfers an association between county averages to individual people. 2. Counties differ in income, school funding, age distribution, and other features; county averages also do not link each person's library use to that person's score. 3. Individual-level data pairing each person's library use with reading outcomes, along with an appropriate design to address confounding, are needed.

a) County-level averages do not establish the same relationship for individuals. b) County differences can confound the relationship, and aggregate averages do not pair individual use with individual scores. c) Collect linked individual-level data and account for confounding or use a suitable experiment.

- Ask whether the proposed outcome could also influence the proposed cause. - Look for a third factor connected to both measured variables. - Match the strength of the conclusion to the observational design.

1. Teenagers experiencing anxiety may turn to screens more often, so anxiety could partly influence screen time. 2. Social isolation, sleep disruption, family stress, or school workload could affect both screen use and anxiety. 3. The survey supports an association at one point in time but does not determine causal direction or isolate a screen-time effect.

a) Anxiety may lead some teenagers to use screens more. b) For example, sleep disruption or social isolation could affect both. c) The survey shows an association, not a causal direction.

- Judge prediction using data not used to fit the model. - A flexible curve can explain both signal and accidental irregularities. - Compare performance on training and test data separately.

1. The quadratic should be preferred because its test error of \(4\) units is much smaller than \(18\) units. 2. A highly flexible model can reproduce random noise and peculiarities of the training data that do not repeat in new data. 3. The degree-9 polynomial is overfit: it has excellent training fit but poor generalization.

a) The quadratic model. b) Perfect training fit may capture noise rather than the underlying relationship. c) Overfitting.

- Notice that selection used an extreme first measurement. - Ask what may happen on a second measurement without any treatment. - A comparison group should experience the same timing and selection process.

1. The players were selected for extreme low results that partly reflect temporary random variation. 2. On a later week, the unusually poor variation is unlikely to repeat as strongly, so percentages can move closer to typical levels even if the routine has no effect. 3. A stronger design randomly assigns similarly low-performing players to the new routine or a comparison routine and compares later changes.

The selected extreme lows can be followed by less extreme results simply because temporary poor variation does not repeat; this is regression toward the mean. Randomly assign comparable low-performing players to treatment and control routines to test the effect.

- Count correct predictions separately for each actual class. - Examine performance on the rare outcome rather than only the total. - A useful metric should reflect the kind of error that matters for the task.

1. The model correctly labels all \(950\) legitimate transactions, so accuracy is \(\frac{950}{1000}=95\%\). 2. It detects \(0\) of \(50\) fraudulent transactions, so its fraud-detection rate is \(0\%\). 3. The classes are highly imbalanced, so a model can obtain high overall accuracy by always predicting the common class while failing completely on the important rare class. 4. The report should include the fraud-detection rate, false-alarm rate, or a full confusion matrix.

a) \(95\%\). b) \(0\%\). c) The common legitimate class dominates the accuracy calculation and hides total failure on fraud. d) Report fraud-detection rate and false-alarm rate, or the full confusion matrix.

- Use actual positives as the denominator for detection and actual negatives for false alarms. - Compare both rates before judging a threshold. - A threshold decision reflects the costs of different mistakes, not only mathematical accuracy.

1. At the low threshold, the detection rate is \(\frac{90}{100}=90\%\), and the false-alarm rate is \(\frac{180}{900}=20\%\). 2. At the high threshold, the detection rate is \(\frac{65}{100}=65\%\), and the false-alarm rate is \(\frac{45}{900}=5\%\). 3. Raising the threshold reduces false alarms but misses more positive cases. 4. The best threshold depends on the relative costs of missed positives, false alarms, follow-up resources, and the intended use.

a) Low: \(90\%\) detection and \(20\%\) false alarms. High: \(65\%\) detection and \(5\%\) false alarms. b) The higher threshold lowers both detections and false alarms. c) The choice depends on the consequences and costs of the two error types.

- Combine group totals rather than averaging the two group means equally. - Weight each mean by its group size. - Ask whether response status is associated with the measured variable.

1. The combined salary total is \(800\cdot62{,}000+200\cdot120{,}000=73{,}600{,}000\) dollars. 2. The full-sample mean is \(\frac{73600000}{1000}=\$73{,}600\). 3. The respondent-only mean is \(73{,}600-62{,}000=\$11{,}600\) too low. 4. Nonresponse is related to salary, so treating the observed salaries as representative of all selected employees is inappropriate.

a) \(\$73{,}600\). b) The respondent-only mean underestimates by \(\$11{,}600\). c) The missingness is related to the outcome, producing nonresponse bias.

- Compare both centers and spreads shown by the boxes. - Consistency is reflected by how tightly outcomes cluster. - One summary statistic cannot determine an entire distribution.

1. Service A has median \(30\) and interquartile range \(32-28=4\) minutes. Service B has median \(25\) and interquartile range \(45-15=30\) minutes. 2. Service A is more consistent because its central half and full whisker range are much narrower. 3. Equal means describe only one measure of center; Service B's broad, skewed distribution can have the same mean while producing far less predictable delivery times.

a) A: median \(30\), IQR \(4\). B: median \(25\), IQR \(30\). b) Service A. c) Reliability depends on variability and distribution shape, not only the mean.

- Randomization creates probabilities, not identical group summaries. - Small samples allow larger chance imbalances. - Evaluate both the assignment procedure and the effect of the baseline difference.

1. Random assignment balances groups on average over many experiments but does not guarantee exact equality in every small sample. 2. Researchers should report baseline differences, check whether they are influential, and consider an analysis of change or an adjustment planned for baseline score. 3. The imbalance alone does not prove manipulation; the assignment procedure should be audited, but chance variation is a plausible explanation.

a) Small randomized groups can differ by chance even under a valid procedure. b) Report and account for the baseline difference when evaluating outcomes. c) No. Unequal means are possible under honest random assignment.

- First recover correctly classified negatives from the false-positive counts. - Use actual positives as the denominator for missed-positive rate. - A shared overall percentage can be built from very different kinds of errors.

1. Group A has \(90\) correct positives and \(90\) correct negatives, so accuracy is \(\frac{180}{200}=90\%\). 2. Group B has \(10\) correct positives and \(170\) correct negatives, so accuracy is also \(\frac{180}{200}=90\%\). 3. Group A misses \(10\) of \(100\) positives, a \(10\%\) missed-positive rate. Group B misses \(10\) of \(20\), a \(50\%\) rate. 4. Different class proportions allow the same total accuracy to hide a much worse error rate on positive cases in Group B.

a) Both groups have \(90\%\) accuracy. b) Group A: \(10\%\); Group B: \(50\%\). c) Accuracy combines error types and is affected by group class proportions, so it can conceal unequal missed-positive rates.

- Compare the conditions in the test set with the conditions in actual use. - A performance estimate is only as representative as the data used to calculate it. - Important operating conditions should appear in validation and in separate checks.

1. The evaluation set does not represent the conditions under which the model will be deployed. 2. Clear weekdays may have more regular traffic and fewer unusual delays, so performance on them can exceed performance during harder weekend and weather conditions. 3. Validation should use held-out trips sampled across the intended deployment distribution, with separate performance summaries by weather, day type, route, and other important conditions.

a) The validation data are unrepresentative of deployment conditions. b) The selected trips are easier and less variable than the full use case. c) Test on held-out data covering all relevant conditions and report subgroup performance.

- Weight each group effect by its share of the population. - An average can combine effects with opposite signs. - Decision-making may require knowing who benefits and who may be harmed.

1. The weighted average effect is \(0.80\cdot10+0.20\cdot(-5)=8-1=7\) units. 2. The positive average hides that the treatment may be harmful for Group 2, so the same conclusion does not apply uniformly to every subgroup. 3. The report should include group-specific effect estimates, uncertainty, group sizes, and evidence about whether the difference between subgroup effects is reliable.

a) \(+7\) units. b) The average conceals a negative estimated effect in Group 2. c) Report subgroup effects with uncertainty and evaluate the evidence for effect differences.

- Calculate change once relative to the original risk and once directly on the percentage scale. - Translate both rates into counts for a common population size. - A relative statement can sound large when the baseline rate is small.

1. The relative reduction is \(\frac{2\%-1\%}{2\%}=\frac{1}{2}=50\%\). 2. The absolute reduction is \(2\%-1\%=1\) percentage point. 3. Without treatment, the expected count is \(10{,}000\cdot0.02=200\); with treatment, it is \(10{,}000\cdot0.01=100\), a difference of \(100\) events. 4. The relative claim is mathematically correct but incomplete without the starting risk and absolute change.

a) \(50\%\) relative reduction. b) \(1\) percentage point absolute reduction. c) About \(100\) fewer events per \(10{,}000\) people. d) The claim is correct but should also report the baseline risk and absolute reduction.

- Identify whether each interval concerns an average or one observation. - Individual outcomes vary even if the mean were known exactly. - Match the interval target to the operational decision.

1. The confidence interval \((490, 510)\) estimates the population or future-period mean demand, while the prediction interval \((420, 580)\) describes uncertainty for one individual future day's demand. 2. Individual days vary around the mean, so uncertainty for one day includes both uncertainty in the mean and natural day-to-day variation. 3. The prediction interval \((420, 580)\) is relevant for planning a single day's staffing.

a) The narrow interval estimates mean demand; the wide interval predicts one future observation. b) A single day has additional natural variability not represented by the confidence interval for the mean. c) Use \((420, 580)\).

- Calculate each change relative to its own starting value. - Compare periods that differ in whether an unusual disruption occurred. - A claim can be numerically correct yet misleading because of endpoint selection.

1. The February-to-March increase is \(\frac{840-600}{600}=0.40=40\%\). 2. The January-to-March change is \(\frac{840-1000}{1000}=-0.16=-16\%\). 3. The year-over-year March change is \(\frac{840-830}{830}\approx0.0120=1.20\%\). 4. The advertised comparison starts from an unusually low outage month and therefore exaggerates ordinary growth; broader and comparable periods give a different picture.

a) \(40\%\). b) January to March: \(-16\%\); year-over-year March: about \(+1.20\%\). c) The selected outage month creates a misleadingly low baseline and overstates growth.

- Compare the center and spread of residuals separately. - Prediction uncertainty should reflect the local size of typical errors. - A single error margin assumes similar variability everywhere.

1. The residuals remain centered near zero, but their spread increases as \(x\) increases. Constant residual variability is therefore questionable. 2. Predictions should be most precise at low \(x\), less precise in the middle, and least precise at high \(x\). 3. A constant \(\pm3\) rule is wider than needed for many low-\(x\) cases but far too narrow for high-\(x\) cases, so it misrepresents changing uncertainty.

a) Residual variability increases with \(x\), so constant spread is questionable. b) Prediction intervals should widen as \(x\) increases. c) One fixed margin ignores the strong change in residual spread across the domain.

- Compare the effect across levels of the environmental condition. - A single effect is questionable when the change depends strongly on another condition. - Context-specific estimates can be more informative than an unweighted average.

1. The report treats a context-dependent effect as one constant effect and therefore misrepresents both dry and wet conditions. 2. The fertilizer effect changes with moisture conditions rather than remaining constant. 3. A more accurate conclusion is that the estimated increase is about \(5\) bushels per acre when dry and \(20\) when wet; any overall average must be weighted by the frequency of those conditions and should not replace the separate effects.

a) The average is incorrectly applied uniformly to two conditions with very different effects. b) The fertilizer effect depends on whether conditions are dry or wet. c) Report the separate \(5\)- and \(20\)-bushel effects and, if needed, a properly weighted overall average.

- Use the performance measure tied directly to the prediction goal. - A complex method should be compared with a simple credible alternative. - Goodness-of-fit summaries and out-of-sample error answer different questions.

1. The seasonal baseline predicts better because its mean absolute error is \(8\) rather than \(12\) units. 2. A baseline shows whether the complex model improves on a readily available prediction rule that captures seasonality. 3. \(R^2\) describes variation explained under a particular fit, while test error directly measures out-of-sample prediction accuracy on the chosen scale; the model performs worse for the stated prediction goal.

a) The seasonal baseline. b) It establishes a meaningful minimum performance standard for the complex model. c) The model's higher-level fit summary does not compensate for larger errors on unseen data.

- Use extreme simulated outcomes divided by all simulated outcomes. - Keep the conditioning direction clear: the model is assumed when the probability is calculated. - Distinguish probability of data under a model from probability of a model given data.

1. The estimated tail probability is \(\frac{12}{1000}=0.012\). 2. If the no-effect model and randomization process were appropriate, about \(1.2\%\) of assignments would produce a mean difference at least as large as \(8.0\). 3. The tail probability is calculated assuming the no-effect model; it is not the probability that the model itself is true.

a) \(0.012\). b) Under the no-effect model, a result at least this large occurs in about \(1.2\%\) of randomizations. c) No. It is a probability of simulated data under the model, not a probability assigned to the model.

- Use the complement of having no false flags in any test. - Count how many opportunities the researchers had to find an unusual result. - Good reporting prevents readers from seeing only the most favorable analysis.

1. The probability of no false flags is \((0.95)^{20}\approx0.3585\). 2. The probability of at least one false flag is \(1-0.3585\approx0.6415\). 3. Searching across many outcomes makes at least one apparently unusual result common even when no outcomes are affected. 4. Researchers should prespecify primary outcomes, report all tested outcomes, or use a method that accounts for multiple comparisons.

a) \(1-(0.95)^{20}\approx0.6415\). b) With twenty opportunities, a small tail probability can arise through selection of the most favorable result. c) Prespecify and report primary outcomes, disclose all tests, or adjust for multiple comparisons.

- Compute rates using applicants within the same department as denominators. - Then recompute after pooling each gender across departments. - Compare how each group is weighted toward the high-rate and low-rate departments.

1. In Department A, the rates are \(\frac{80}{100}=80\%\) for women and \(\frac{18}{20}=90\%\) for men. In Department B, they are \(\frac{2}{20}=10\%\) for women and \(\frac{60}{100}=60\%\) for men. 2. Overall, women have \(\frac{82}{120}\approx68.3\%\), while men have \(\frac{78}{120}=65\%\). 3. More women applied to high-admission Department A, while more men applied to low-admission Department B. Different group weights reverse the aggregated comparison.

a) Men have higher admission rates in both departments: \(90\%\) versus \(80\%\) in A and \(60\%\) versus \(10\%\) in B. b) Overall, women have about \(68.3\%\) admitted and men \(65\%\). c) The applicant groups are distributed very differently across departments with very different base admission rates.

- Random assignment balances groups before later events occur. - Ask who is missing from each final comparison and why. - A trustworthy report should show how different reasonable assumptions about missing outcomes affect the conclusion.

1. Differential dropout can destroy the comparability created by random assignment among the participants who remain. 2. If exercise participants with little improvement or difficulty following the program are more likely to leave, the completers can make the program look more effective than it was for everyone assigned. 3. The report should compare dropout reasons and baseline traits, include outcomes for all assigned participants when possible, and show how conclusions change under reasonable assumptions about the missing outcomes.

a) Unequal attrition can make the remaining groups systematically different. b) For example, exercise participants who do not improve may be more likely to drop out. c) Analyze all assigned participants when possible and report how the conclusion changes under reasonable assumptions about missing outcomes.

- Separate goodness of fit from evidence about cause. - Ask what third variable changes systematically with both annual series. - Account for the shared time trend before interpreting the relationship.

1. \(R^2\) describes the strength of a linear association; it does not establish a causal mechanism or rule out confounding. 2. Both variables have strong upward time trends, so later years are high in both series even if the variables are unrelated. 3. A better analysis would account for time trend and inflation, compare year-to-year changes after adjusting for inflation and use predictors connected to a plausible textbook-price mechanism.

The high \(R^2\) can be caused by a shared upward time trend and does not establish causation. A more meaningful analysis should account for the time trend and inflation, then evaluate variables tied to a plausible pricing mechanism.

- An estimate from one sample is not expected to match another exactly. - Compare both the centers and the full plausible ranges. - Investigate methodological differences before making claims about conduct.

1. Different random samples and study conditions can produce different estimates; disagreement does not by itself establish misconduct. 2. Study 1 supports positive effects from \(2\) to \(8\), while Study 2 includes negative, zero, and small positive effects from \(-4\) to \(2\). They share the boundary value \(2\) but give substantially different centers. 3. Researchers should compare protocols, populations, measurements, and attrition, and combine or repeat well-designed studies when appropriate.

a) Sampling variation or design differences can explain disagreement; the intervals alone do not show fraud. b) The first supports a positive effect, while the replication supports values from moderately negative through small positive. c) Audit design differences and conduct or combine additional carefully designed replications.

- Compare the predicted percentage with the observed relative frequency. - State both the direction and size of the discrepancy. - Probability accuracy and ranking performance are different model properties.

1. The observed event rate is \(\frac{90}{200}=0.45=45\%\). 2. The observed rate is \(45\%-30\%=15\) percentage points higher than predicted, so the model underpredicts risk for this group and is poorly calibrated here. 3. Calibration compares predicted probabilities with observed frequencies; ranking ability requires comparing whether cases with higher assigned risks actually tend to experience more events than cases with lower assigned risks.

a) \(45\%\). b) The model underpredicts by \(15\) percentage points for this group. c) Calibration within one risk group does not measure how well the model orders cases across different risk levels.

- Count how many subgroup opportunities were examined before one was highlighted. - Timing matters: ask whether the subgroup was chosen before or after seeing results. - Exploratory findings become stronger when independently replicated.

1. Searching many subgroups creates multiple opportunities for a chance result, and the subgroup was selected after inspecting the data. 2. The claim would be stronger with a planned subgroup comparison and replication in an independent study designed to include that subgroup. 3. A prespecified analysis is planned before outcomes are examined; a data-discovered analysis is exploratory and requires more cautious interpretation and confirmation.

a) The result may be a chance finding from multiple after-the-fact subgroup searches. b) Use a prespecified subgroup comparison and independent replication. c) Prespecified analyses test planned questions; data-discovered analyses generate hypotheses that need confirmation.

- Use falsely stopped simulations divided by all no-effect simulations. - Compare both the absolute and multiplicative increase over the target rate. - A decision rule includes when the data are examined and when collection stops.

1. The simulated false-positive rate is \(\frac{142}{1000}=0.142=14.2\%\). 2. This is \(14.2\%-5.0\%=9.2\) percentage points above the intended rate and is about \(\frac{14.2}{5}=2.84\) times as large. 3. Each additional look creates another opportunity for random fluctuation to cross the threshold; the original tail-probability interpretation assumes a fixed analysis plan.

a) \(14.2\%\). b) It is \(9.2\) percentage points higher, about \(2.84\) times the intended rate. c) Repeated optional checking adds chances to stop on a random extreme result.

- Compute each location's change using the same subtraction order. - Subtract the comparison change from the treated change. - Use the untreated location to estimate what might have happened anyway.

1. The treated city changes by \(70-100=-30\) crashes. The comparison city changes by \(88-110=-22\) crashes. 2. The adjusted difference in changes is \((-30)-(-22)=-8\) crashes. 3. A broad trend reduced crashes even without the law, as shown by the comparison city. The raw decline attributes that shared trend to the law, while the adjusted comparison suggests an additional decline of about \(8\) crashes.

a) Treated city: \(-30\); comparison city: \(-22\). b) \(-8\) crashes. c) Most of the decline may reflect a trend affecting both cities rather than the law alone.

- Identify which observations share the same source. - Repeated measurements do not add as much new information as measurements from new independent units. - Either change the unit of analysis or use a model that represents the grouping.

1. Observations from the same student are likely correlated, violating the assumption that all \(300\) observations are independent. 2. Treating repeated observations as independent overstates the amount of separate information and usually makes standard errors and intervals too small. 3. The analysis can summarize each student first or use a method that treats observations from the same student as related; the design can also sample more students rather than only more days per student.

a) Independence is violated within each student's repeated measurements. b) Uncertainty is understated because the effective sample size is less than \(300\). c) Analyze student-level summaries or use a method that accounts for repeated observations from each student, and increase the number of students when possible.

- Consider what happens when many new values are placed exactly at the center. - A filled-in value is not the same as an observed value. - Valid uncertainty should include uncertainty about what was missing.

1. Replacing missing values by the observed mean leaves the completed-data mean at \(75\) if the observed mean was \(75\). 2. Every imputed value lies exactly at the center, so the completed data have artificially reduced variability. 3. The method treats guessed values as known and ignores uncertainty about missing outcomes, causing model uncertainty and confidence-interval width to be understated.

a) It preserves the observed mean of \(75\). b) It artificially reduces variability by adding many central values. c) The analysis ignores uncertainty in the imputed values and therefore appears too precise.

- Ask when each predictor becomes available relative to the prediction decision. - A held-out set is only useful if it represents the information available in real use. - Validation should preserve time order when the application predicts the future.

1. The predictor contains information recorded after the outcome and therefore leaks the answer into the model. 2. Both randomly split sets contain the leaked variable, so the model can exploit future information in training and testing even though that information is unavailable at prediction time. 3. Predictors must be restricted to information available before the exam, and validation should reproduce the intended timeline, such as training on earlier terms and testing on a later term.

a) Outcome or future-information leakage. b) The leaked variable appears in both random splits, so the test is still unrealistic. c) Use only pre-exam predictors and validate on data separated according to the real prediction timeline.

- A model's evidence is tied to the population and period on which it was evaluated. - Compare current inputs and errors with the development data. - Model evaluation should continue after deployment rather than occur only once.

1. The population and relationship between predictors and outcomes may have shifted over time, so the 2018 validation distribution no longer represents 2026 applications. 2. The lender should check current calibration, error rates, subgroup performance, predictor distributions, and whether outcome definitions or economic conditions changed. 3. The model should be recalibrated, retrained, replaced, or restricted after validation on recent representative data, with continuing performance monitoring.

a) The applicant population and the relationships used by the model have changed, so the old validation is no longer reliable. b) Check current calibration, error rates, subgroup results, and predictor shifts. c) Revalidate on recent data and recalibrate, retrain, replace, or limit the model as needed.

- Keep the groups defined by the random mechanism. - A choice made after assignment can be related to the outcome. - Distinguish the effect of being offered a program from the effect among self-selected users.

1. The assignment-based effect estimate is \(74-70=4\) units. 2. People who choose to participate may differ in motivation, availability, or baseline condition from nonparticipants and controls, so the \(78-70=8\) comparison mixes treatment with self-selection. 3. Comparing all people according to original assignment preserves randomized group comparability and estimates the effect of offering or assigning the program.

a) \(4\) units. b) Actual participation is self-selected after assignment, so the \(8\)-unit comparison is confounded. c) Compare all \(200\) people by their original randomized assignment.

- Begin by splitting the population according to the base rate. - Apply each conditional positive rate within the appropriate branch. - Check which event appears after the conditioning bar in each probability.

1. Of \(10{,}000\) people, \(200\) have the condition and \(9800\) do not. 2. The expected true positives are \(200\cdot0.90=180\). The expected false positives are \(9800\cdot0.05=490\). 3. The percentage with the condition among all positive tests is \(\frac{180}{180+490}\approx0.2687=26.87\%\). 4. Sensitivity conditions on actually having the condition, while the requested probability conditions on receiving a positive test; the low base rate and false positives matter.

a) \(180\) true positives and \(490\) false positives. b) About \(26.87\%\). c) Sensitivity is \(P(+\mid C)\), not \(P(C\mid +)\); the population base rate changes the latter probability.

- Track whether information from the supposed test set changed any modeling decision. - Selecting the maximum among many noisy estimates favors unusually high values. - Reserve some data until every model choice is complete.

1. Repeatedly using test results to choose among models makes information from the test set part of model selection. 2. The highest observed test accuracy includes favorable random variation, so it is likely biased upward relative to future performance. 3. Use training data for fitting, separate validation data for model choices, and a final untouched test set for one-time evaluation.

a) Test performance influenced model selection, so the set is no longer independent of development. b) The reported accuracy is likely too high. c) Separate training, validation, and final untouched testing stages.

- Ask whether one participant's assigned treatment can change another participant's experience. - Consider what happens when the control group receives part of the active program. - Choose assignment units that reduce contact across treatment conditions.

1. The study's assumption that one group's treatment does not affect outcomes in the other group is threatened by spillover or interference. 2. Control students may receive part of the tutoring benefit, making the groups more similar and often reducing the estimated difference; other social effects could also complicate direction. 3. Researchers could assign more widely separated schools or groups, measure how much sharing occurs, and report how the results change when cross-group exposure is considered.

a) Spillover makes the treatment and control conditions overlap. b) Shared materials can contaminate the control condition and usually shrink the observed contrast. c) Assign more separated groups and measure or account for cross-group exposure.

- Separate how treatment was assigned from how outcomes are later analyzed. - Matching is useful when partners share variation that can cancel in a difference. - Use the same observational unit in the analysis that the randomization created.

1. The original within-pair random assignment still occurred, so discarding labels does not erase the random treatment mechanism or necessarily change the treatment-group mean difference. 2. Matching made partners similar at baseline, and the within-pair differences vary by only \(4\) points compared with \(15\)-point variation among individual outcomes. Ignoring pairs fails to remove that shared baseline variation and therefore loses precision. 3. The analysis should use the \(50\) within-pair differences, centered at the observed mean difference of \(3.2\) points, with a method designed for paired data.

a) No. Treatment was still randomly assigned within pairs, although the analysis ignores useful design information. b) Ignoring matching leaves large between-participant variation in the error instead of canceling it within pairs. c) Analyze the \(50\) A-minus-B pair differences, whose mean is \(3.2\) points.

- Separate uncertainty inside one chosen model from uncertainty about choosing the model itself. - Good historical fit does not make two different future assumptions equivalent. - Show how the conclusion changes under other plausible model assumptions.

1. The report omits model uncertainty: different plausible model structures and assumptions produce very different forecasts. 2. A narrow interval conditional on Model A can describe random error within that model while ignoring the larger uncertainty about which model is appropriate. 3. A responsible report should show forecasts from both justified models, explain their assumptions, evaluate them on relevant validation data, and show how the forecast changes under the different plausible models.

a) Uncertainty about which model and assumptions are appropriate. b) It can look precise while ignoring another plausible forecast far from \(120\). c) Report alternative models, assumptions, validation performance, and sensitivity of conclusions to model choice.

- Evaluate random assignment, sampling, missing outcomes, and uncertainty separately. - Distinguish a sample average from a population effect and from every individual's response. - A strong conclusion should state both the supported effect and the study's scope limitations.

1. Random assignment initially supports causal inference, and the small tail proportion indicates that the observed completer difference is unusual under the no-effect randomization model. However, differential attrition weakens the randomized comparison, so the completer-only result is not a clean causal estimate. 2. Volunteers from one fitness center are not a random sample of all adults, so broad population generalization is unsupported. 3. The \(4.0\) value is a sample mean difference, not an exact population effect; the confidence interval shows plausible mean effects from \(0.5\) to \(7.5\). An average effect also does not imply the same change for every individual. 4. The much higher dropout in the drink group can make completers systematically different and weaken the comparability created by random assignment. 5. A supported conclusion is that the study found a higher mean heart rate among drink-group completers and statistical evidence inconsistent with a no-effect model, but the effect size is uncertain, differential attrition weakens causal interpretation, and the result cannot be generalized to all adults.

a) Random assignment initially supports causation, but differential attrition means the completer-only comparison is not a clean causal estimate. b) The volunteer sample from one fitness center does not support generalization to all adults. c) \(4.0\) is an estimate with interval \((0.5, 7.5)\), and a mean effect is not an identical individual effect. d) Unequal dropout can make the remaining randomized groups systematically different. e) The drink-group completers had a higher mean heart rate, but the effect size is uncertain, differential attrition weakens causal interpretation, and generalization to all adults is unsupported.