- Use the random sample to identify the population. - Determine whether any explanatory treatment was assigned. - Separate estimating a rate from explaining why it occurs.

1. Random selection permits estimation of the defect proportion for all items produced during that shift. 2. The study is descriptive and observational. 3. It can support generalization about the production population but no causal conclusion about why defects occurred.

The sample defect rate may be generalized to estimate the defect proportion among all items produced during that shift. The study does not identify causes of defects.

- How were the apps assigned to participants? - Did the researchers randomly select the participants, or did they join themselves? - Consider causation and population representation separately.

1. The app assignments are random, so the experiment supports a causal comparison between the apps among the student volunteers. 2. The students joined by volunteering, not by random selection from all students. 3. The study design therefore does not support automatic generalization to the wider student population.

Yes, random assignment supports a causal comparison for the volunteers. No, volunteering does not make them a random sample of all students.

- How did people enter the listening experiment? - Was the random step about choosing participants or choosing hearing protection? - What claim requires a representative sample rather than a fair treatment comparison?

1. Random assignment helps compare the two protectors as possible causes of differences in hearing clarity among participants. 2. Representing adults across the country would require an appropriate random sample of that population, which volunteering did not provide. 3. The organizer has confused random treatment assignment with random participant selection.

The organizer confused random assignment with random sampling. Assignment supports a causal comparison for volunteers; it does not make them representative of all adults.

- Which step decides who enters the study? - Which step decides the practice-sheet format? - Match each step to generalization or causation.

1. Random selection helps the sample represent students on the enrollment list. 2. Random assignment permits a causal comparison of the two practice-sheet formats. 3. Whether one format actually works better would still depend on the observed results.

Yes. Random selection supports generalization to students represented by the list, and random assignment supports comparing the formats as possible causes of score differences.

- Who has a name on the list used for the random selection? - Could a student who is not in any club be selected? - Does a fair draw from one group always represent a larger group?

1. The random selection includes only students on the club-member list. 2. Students who belong to no club have no chance of being chosen. 3. The sample can help represent club members, but not automatically every student at the school.

No, not from the sampling method alone. It can represent students who belong to an after-school club, but not automatically all students at the school.

- Could a first-year student who was not selected still be part of the group the university wants to understand? - Imagine two selected students who receive different messages. Why is the way those messages were chosen important? - Does a sound plan tell you which email worked before you see appointment results?

1. The random sample allows the study to represent first-year students at that university. 2. Random assignment allows a fair causal comparison of the two email designs. 3. A real effect would require evidence from the results; randomization alone does not prove one design works better.

Random sampling supports generalizing to the university's first-year students. Random assignment supports testing whether the email design changes appointment-scheduling rates; it does not by itself prove that one email is better.

- Use the sampling method to determine the population scope. - Decide whether attendance was imposed as a treatment. - Phrase the relationship as association unless causation is justified.

1. Random selection permits generalization to the population of registered voters in the county. 2. Meeting attendance was observed, not assigned. 3. The result supports an association between attendance and bond support in that population. 4. It does not show that attending a meeting causes support.

The county may generalize the association to all registered voters in the county. It may not conclude that attending a public meeting causes greater support for the bond.

- Was program assignment made by chance? - Which ages and locations are actually represented? - Is a difference between sample averages the same as a proven treatment effect?

1. Random assignment supports a causal comparison of the two programs among the recruited adults. 2. The lower observed group mean does not, by itself, establish that tai chi truly prevents more falls. 3. Participants came from two centers and were not randomly sampled from all older adults.

The design supports comparing the programs as possible causes of differences in falls among these participants. The observed difference alone does not prove a benefit, and the results cannot automatically be generalized to all older adults.

- Which boards had a chance to enter the random sample? - What decision was made by chance after the boards were chosen? - Is being flagged for repair necessarily the same as having a real defect?

1. Random selection supports generalization to the circuit boards made on Monday. 2. Random assignment lets the company compare how often a board is flagged for repair under each inspection method. 3. A flag is not proof of a real defect. Without checking which boards truly have defects, the study cannot compare the methods' accuracy at finding real defects.

It can represent Monday's production and study how the assigned inspection method affects the proportion of boards flagged for repair. It cannot tell which method finds more actual defects without independently checking the boards.

- Define the population using both place and time. - Use the sampling frame, not the broader species name, to set the boundary. - Avoid extending the result beyond units that could have been selected.

1. The random sample represents deer in the defined sampling population: the protected area during winter under the observation frame. 2. It does not automatically represent deer in other areas or seasons. 3. The estimate may be generalized only to the population from which the random sample was selected.

The estimate may be generalized to deer in that protected area during the winter sampling period that were included in the observation frame. It should not automatically be generalized to unobserved deer, other areas, other seasons, or other deer populations.

- Identify exactly who could be selected. - Compare the sampling frame with the population named in the proposed conclusion. - Limit generalization to the frame represented by the random sample.

1. The random sample represents the population on the membership list. 2. Residents who are not association members had no chance of selection. 3. Therefore, the result may be generalized to association members but not automatically to all neighborhood residents.

No. The result may be generalized to association members, not to all neighborhood residents, because nonmembers were outside the sampling frame.

- Which part of the headline is a claim about what happened, rather than what the study could test? - Think about the places in the state that had no chance to enter this study. - How is comparing two signals different from knowing which one produced fewer late crossings?

1. Random assignment gives the city a fair way to investigate whether the signal changes late-entry counts. No results are given, so the design alone cannot show that countdown signals stop or reduce late crossings. 2. The sample came from intersections in one city, not a random sample of intersections throughout the state. The design alone does not support a statewide claim.

First, the study design does not tell us that countdown signals reduce late crossings; we would need the results. Second, only intersections in the city were sampled, so the study cannot automatically speak for the whole state.

- Which actual treatments could you compare using the recorded reaction times? - How would you find out what happened without the supplement? - Would comparing the two doses answer that second question?

1. Random assignment supports a causal comparison between the two tested doses among participants. 2. Without a no-supplement or placebo group, the study cannot establish how either dose compares with taking no supplement.

The study can investigate a causal difference between the high and low doses. It cannot compare either dose with taking no supplement because no group received that condition.

- What age group was eligible for the sample? - Which step assigned patients to the scheduling systems? - Could patients under \(65\) have been selected?

1. Random sampling supports conclusions about the center's patients age \(65\) or older in its sampling frame. 2. Random assignment supports a causal comparison of the scheduling systems on missed-appointment rates. 3. Patients younger than \(65\) are outside the sampling frame. 4. The design does not itself show that one system reduces missed appointments.

The study can compare scheduling systems causally and can represent the center's patients age \(65\) or older. It cannot automatically generalize to younger patients or claim a benefit without studying the results.

- Which teachers appeared on the sample roster? - What did the researchers assign at random? - Could a teacher outside the network have been selected?

1. Random assignment permits a causal comparison of the formats on the assessment scores. 2. Random sampling represents the teachers in this charter-school network. 3. Teachers outside the network were not on the sampling list, so they were not represented by chance selection.

The study can compare training formats causally for teachers in the network. It does not automatically represent all U.S. teachers because teachers outside the network were not sampled.

- Could someone who is not one of the publisher's current remote proofreaders be drawn for the study? - Why is it helpful that everyone checks the same passage for the same length of time? - What different length of use would require another study?

1. Random sampling represents the publisher's current remote proofreaders. 2. Random assignment permits a causal comparison of headphones and usual equipment on the number of missed errors in the same timed passage. 3. The experiment lasted one week and measured performance on Friday, so it cannot establish what would happen after months of use.

The study could represent the publisher's current remote proofreaders. It could investigate whether headphones affect the number of errors missed on the Friday test. The one-week design does not tell us what would happen after months of use.

- Think about things that differ between commuters besides whether they bike. - Could the two groups differ in ways that also affect travel time? - What would the study need to know before saying what happens when someone switches travel modes?

1. People who bike may live closer to work than people who drive; differences in journey length could explain the mean-time difference. 2. Nobody was assigned a travel mode at random, so the observed group difference may be due to other differences between the commuters. 3. The sample can describe a relationship, but it cannot show what would happen if a driver changed to biking.

For example, cyclists might live closer to work. Because travel mode was observed rather than assigned, this difference could explain the shorter commutes. The study does not show that switching a driver to biking would shorten that person's trip.

- Think about who had a chance to be included in this survey. - Could something besides coffee help explain a person's sleep hours? - What can be said about the people who actually answered without claiming more?

1. The reporter chose friends, not a random sample of adults. The pattern may not hold among adults generally. 2. Nobody was assigned to drink coffee. Other differences, such as work hours, could affect both coffee drinking and sleep. 3. The survey does support describing the reported difference among these \(12\) friends.

The friends do not represent all adults through a random sample, and coffee was not assigned, so the survey cannot show cause and effect. It does show that coffee drinkers among these \(12\) friends reported less sleep than the others.

- Did the city assign some neighborhoods to plant trees? - Could another difference between neighborhoods affect their temperatures? - How could a headline describe the recorded relationship without claiming a cause?

1. Tree coverage was measured, not randomly assigned. 2. The study found an association between coverage and average temperature. 3. Other neighborhood differences may explain some or all of that relationship.

Tree coverage was observed, not assigned, so other neighborhood differences could help explain the cooler temperatures. One suitable headline is: “Neighborhoods with more tree coverage tend to have lower summer temperatures.”

- Which branch supplies the names for the random sample? - Which branch is excluded before any assignment occurs? - Which random step addresses comparing treatments and which addresses representing students?

1. Random assignment of reminders supports a causal comparison of on-time homework submission for sampled app users. 2. The sampling frame contains only active app users. 3. Students who do not use the app cannot be selected, so the sample does not necessarily represent all district students.

Random assignment supports a causal comparison of the two app reminders among sampled active users. The findings do not automatically generalize to nonusers because they are absent from the sampling frame.

- Separate how the observational units were selected from how the explanatory variable arose. - Decide whether the study supports population generalization. - Decide independently whether it supports causation.

1. Random selection permits generalization from the sampled receipts to all purchases made by the chain during that month. 2. Waiting time was observed rather than randomly assigned. 3. The study supports an association but not a cause-and-effect conclusion.

The chain may generalize the observed association to all of its purchases during that month. It cannot conclude that longer waits cause lower satisfaction because there was no random assignment of waiting times.

- What does random assignment contribute to a study? - Was the fitness-center group randomly selected from all adults? - Does a larger observed sample mean by itself prove the treatment helps?

1. Random assignment allows the plans to be compared as possible causes of differences in endurance improvement among the participants. 2. A higher observed group mean alone does not establish that the new plan has a real positive effect; chance variation is possible. 3. The participants volunteered at one fitness center, so they do not form a random sample of all adults.

The randomized design supports a causal comparison of the two training plans among these volunteers, though an observed difference alone does not prove an effect. The study cannot automatically generalize its findings to all adults.

- Decide whether the data came from a sample or the entire stated population. - Ask whether the explanatory variable was imposed by researchers. - Keep description of a relationship separate from a causal claim.

1. Because data were collected for every player in that league season, the result describes that full population rather than estimating from a sample. 2. Playing time was not randomly assigned as a treatment. 3. The observed relationship is an association and does not establish that additional minutes cause injuries.

The association applies to all players in that league during the season studied because the data form a census of that population. A cause-and-effect conclusion is not justified.

- Examine how participants entered the sample. - Identify the group for which data were actually collected. - Ask whether either variable was imposed as a treatment. - State generalization and causation limits separately.

1. The sample was a convenience sample of willing people at one location. 2. The result cannot be generalized to all city residents, all farmers-market customers, or another defined population. 3. The variables were observed rather than assigned. 4. The study supports an association only among the sixty people interviewed, not a cause-and-effect conclusion.

The study supports an association only among the sixty willing people interviewed at that market. It does not support statistical generalization to a broader population or a claim that buying local food causes greater climate concern.

- Identify whether students, classes, or schools were assigned by chance. - Did the six schools enter the study through random selection? - Think separately about a valid causal comparison and the population represented.

1. Classes, not individual students, are randomly assigned within each school. 2. That assignment supports a causal comparison of routines among participating classes; a higher observed mean alone does not prove a positive effect. 3. The schools volunteered rather than being randomly sampled, so the study does not automatically represent other high schools.

The study can compare the routines causally for participating classes, but the observed increase alone does not prove a real benefit. Volunteer schools do not support statistical generalization to all high schools.

- Use the household selection method to determine the population. - Ask whether solar adoption was imposed by researchers. - Allow for alternative explanations when treatment is self-selected.

1. Random sampling supports generalization to households in the county. 2. Solar adoption was self-selected, not randomly assigned. 3. The study supports an association between solar adoption and electricity bills in the county. 4. It does not establish that installing solar panels caused the full observed difference.

The association may be generalized to county households. A causal conclusion is not justified because households chose whether to install solar panels, so other differences may explain part of the bill difference.

- Use the random sample to identify the population. - Determine whether the exposure was imposed or found in past records. - Account for alternative explanations when stating the relationship.

1. Random sampling supports generalization to adults in the county. 2. The study is retrospective and observational because past exposure was not assigned. 3. The result supports a countywide association. 4. It does not establish that pollution exposure caused the diagnoses because confounding variables may exist.

The association may be generalized to adults in the county. The retrospective observational design does not justify a cause-and-effect conclusion.

- What is selected and assigned: individual employees or whole teams? - Which teams could appear on the company's roster? - Does random assignment permit comparison or guarantee a benefit?

1. Work teams are both the sampling and treatment-assignment units. 2. The random sample supports generalization to the delivery service's U.S. dispatch teams on its roster. 3. Random assignment allows a causal comparison of the two scheduling methods at the team level. 4. The design alone does not prove that either method raises the proportion of deliveries that arrive on time.

Results can represent the delivery service's U.S. dispatch teams on its roster. The design supports a causal comparison of digital-board and paper scheduling at the team level, without itself proving either works better.

- Does the report count crashes or measure how much cycling happens on each kind of street? - Would a street with more bicycle traffic have more chances for crashes? - Was the presence of bike lanes randomly assigned?

1. The records cover reported bicycle crashes in that city and year, not unreported crashes. 2. The count comparison tells where more reported crashes occurred. 3. Without numbers of bike trips or miles ridden on each type of street, crash counts cannot show which street type has a higher crash risk per trip or mile. 4. Bike-lane status was not assigned randomly, so a causal claim is not justified.

More reported crashes occurred on streets without bike lanes in that city and year. The data do not establish greater crash risk per rider or trip or show that bike-lane absence caused crashes. Other years, cities, and unreported crashes are outside these records.

- Which units received the irrigation schedules by chance? - Did the farms enter through a random sample? - Separate the question the experiment can test from a proven result.

1. The randomized units are plots within the participating farms. 2. Random assignment supports a causal comparison of watering times for these plots, but the observed yield difference alone does not prove morning watering is better. 3. The farms volunteered, so they were not randomly sampled from all farms.

Watering times can be compared causally for plots in the participating farms. The observed higher mean does not alone prove a benefit, and the design does not support automatic generalization to all farms.

- Separate how the counties entered the study from how the alert systems were chosen. - Ask whether random selection supports generalization. - Ask whether the explanatory condition was imposed by the researchers.

1. The counties were randomly selected, so the observed association may be generalized to counties in the state. 2. The alert systems were not randomly assigned; the agency observed systems already in use. 3. Therefore, the study supports an association between phone alerts and shorter median evacuation times for counties in the state, but it does not establish that phone alerts caused the difference.

The results support a statewide association: counties using phone alerts tend to have shorter median evacuation times. Because the alert systems were not randomly assigned, a cause-and-effect conclusion is not justified.

- What group of batteries was sampled? - Which condition was assigned by chance? - What time point was measured, and what time points were not?

1. Random sampling allows conclusions about batteries from that production lot. 2. Random assignment allows a causal comparison of the two charging protocols for remaining capacity after \(100\) cycles. 3. The experiment does not measure performance after many more cycles or over the full life of the batteries.

The study can represent the production lot and compare the protocols causally for remaining capacity after \(100\) cycles. It cannot establish which is better over the entire battery life without longer testing.

- Analyze participant selection separately from treatment assignment. - Connect random selection with the population represented. - Connect random assignment with whether the explanatory condition can be treated as a cause.

1. Study I uses random selection, so its observed association may be generalized to all customers. Because app use is self-selected, it does not establish causation. 2. Study II uses random assignment, so it can establish a causal effect of receiving the app among the participants. Because the participants were volunteers rather than a random sample, it does not support statistical generalization to all customers.

Study I: generalization to all customers is supported; causation is not. Study II: causation is supported for the participants; generalization to all customers is not.

- Define the population from the exact list used for random selection. - Notice which calls never had a chance to appear in that list. - Ask whether the explanatory condition was imposed by the researchers.

1. The sampling frame contains completed customer-support calls in June, so random selection supports generalization to that population of calls. 2. Calls that were abandoned or never completed are outside the sampled population. 3. Interpreter use was observed rather than randomly assigned. 4. Differences in issue complexity, language needs, or customer circumstances could explain the longer times, so causation is not supported.

The association may be generalized to completed customer-support calls in June, not to abandoned or other calls outside that frame. Because interpreter use was not randomly assigned, the study does not show that using an interpreter caused longer resolution times.

- Which behavior was actually recorded for each customer? - What did the retailer assign by chance? - Could a result about opening emails answer a question about purchases?

1. Random assignment supports a causal comparison of the subject lines on email opening. 2. Random sampling supports generalization to customers represented by the retailer's list. 3. Purchases were never measured, so the experiment cannot establish an effect on buying behavior.

The retailer can study whether subject-line wording causes a difference in email-open rates among its customers. It cannot claim a change in sales because purchases were not measured.

- Define the population using the time period in the sampling frame. - Determine whether the explanatory variable was assigned or merely recorded. - Consider whether a later system change makes the old population comparable to the new one.

1. Random selection supports generalization to the 2025 user sessions represented by the sampling frame. 2. Session entry method was observed rather than randomly assigned, so only an association is supported. 3. The redesign may have changed user behavior and the meaning of the entry categories. 4. Therefore, the result should not automatically be generalized to 2026 sessions under the redesigned system.

The study supports an association between entry method and session duration for the 2025 sessions represented by the sample. It does not establish causation and should not automatically be generalized to sessions after the 2026 redesign.

- Is this a sample of current stores or all of them? - How were the inventory systems assigned? - What could differ between current and future locations?

1. All \(12\) current stores participate, so there is no sampling uncertainty about which current stores are included. 2. Random assignment allows a causal comparison of inventory systems for those stores under the study conditions. 3. The study does not establish a benefit without evidence from the results. 4. Future stores or stores in other regions were not sampled and may differ from the current stores.

The design allows a causal comparison of the inventory systems for the company's \(12\) current stores. A result for these stores would not automatically apply to future stores or other retailers.

- Check whether the policy was assigned by a random mechanism. - Check whether the states were sampled from a larger population. - Keep the conclusion limited to the states and period actually observed.

1. The policy condition was not randomly assigned, so differences between the states provide alternative explanations for the crash-rate change. 2. The study does not establish that the law caused the larger decrease. 3. Because only these two deliberately chosen states were studied, the result does not support broad statistical generalization to other states. 4. The appropriate conclusion is a descriptive association for the two states during the studied period.

The study shows that the adopting state had a larger crash-rate decrease than the comparison state during the studied period. It does not establish causation and does not support generalization to other states.

- Use the sampling process to set the population scope. - Ask whether researchers imposed the explanatory condition. - Phrase the conclusion as a relationship rather than an intervention effect.

1. Random selection supports generalizing the association to adult patients in the health-system population represented by the sampling frame. 2. The genetic marker was observed rather than assigned by researchers. 3. The study supports an association between the marker and condition prevalence. 4. It does not show that changing or removing the marker would cause a change in the condition, and other biological factors may explain the relationship.

Among adult patients represented by the sampling frame, the genetic marker is associated with higher prevalence of the condition. The observational study does not establish a cause-and-effect relationship.

- Which customers are actually in the sampling list? - What was assigned by chance after customers were selected? - Could customers using other languages appear in this sample?

1. Random assignment allows a causal comparison of the two reminder messages on on-time payment. 2. Random sampling supports representing customers in the English-language communication frame. 3. Customers who selected other languages were not eligible for selection, so generalization to all customers is not automatic. 4. Randomization by itself does not show one version works better.

The store can compare the two reminder messages causally on on-time payment for customers in the English-language sample frame. It cannot automatically apply the result to customers who selected other languages.

- What did the software measure after each safety lesson? - Which step chose the participants, and which step assigned the lessons? - Does a virtual exit measure exactly how someone acts in a real emergency?

1. Random sampling can help the participants represent factory employees on the selection list. 2. Random assignment allows a causal comparison of the guide and video on time to reach the virtual exit. 3. These times come from a simulation. No one was observed evacuating in a real emergency.

The study can compare whether the written guide or video changes simulated evacuation time for factory employees represented by the sample. It cannot establish what would happen in a real emergency.

- Is the outcome a statement of intention or a completed vote? - Which part of the study is randomized? - Keep the possible conclusion tied to what each respondent actually reported.

1. Random assignment supports a causal comparison of descriptions on stated voting intentions. 2. Random sampling allows appropriate conclusions to represent the voter population in the sampling frame. 3. The study records intentions, not ballots cast during an election; actual votes are unmeasured. 4. Assignment alone does not prove the descriptions make intentions different.

The study can compare the descriptions causally on stated voting intention among the sampled voter population. It cannot establish an effect on actual votes, because no votes were measured.

- Which seed variety was eligible for selection? - What condition was assigned by chance? - Which growing conditions stayed fixed during the study?

1. Random selection represents seeds of the sampled variety in the current inventory. 2. Random assignment allows a causal comparison of coatings on germination under the tested greenhouse conditions. 3. Other tomato varieties and different growing conditions are outside the tested setting. 4. The design does not by itself prove that the new coating improves germination.

The study can compare the coatings causally for seeds of this tomato variety in the company's current inventory under the tested conditions. A result cannot automatically be applied to other varieties or growing conditions.

- Decide whether the study used a sample or included the entire stated group. - Ask whether the explanatory condition was assigned by chance. - Keep the conclusion within the location and time period actually observed.

1. Because every 2025 call at the airport was included, the observed association describes that finite set of calls without sampling uncertainty. 2. Staffing was not randomly assigned, so other factors such as crowd size or incident severity may be related to both staffing and response time. 3. The study does not establish causation and does not automatically generalize to other airports or years.

For this airport’s 2025 calls, higher staffing was associated with shorter response times. The study cannot show that higher staffing caused the shorter times, and it does not automatically support conclusions about other airports or years.

- Identify the exact sampling frame. - Decide whether the thermostat setting was imposed or merely observed. - Keep the conclusion within the technology and time period represented in the data.

1. Random selection supports generalization to the utility’s customer homes with smart thermostats. 2. Thermostat settings were observed rather than randomly assigned, so the study supports association but not causation. 3. The conclusion does not automatically extend to homes without smart thermostats or to seasons other than July 2026.

Among the utility’s customer homes with smart thermostats, lower daytime thermostat settings were associated with greater electricity use during July 2026. The study does not establish causation and does not automatically apply to homes without smart thermostats or to other seasons.

- Separate the role of random selection from the absence of random assignment. - Look for pool characteristics that could affect both staffing and rescues. - Avoid interpreting the direction of an association as the direction of a treatment effect.

1. Random selection supports generalizing the observed association to public swimming pools in the state during the summer studied. 2. Lifeguard staffing was not randomly assigned, so the study does not support causation. 3. Pools with stronger currents, larger crowds, or higher-risk activities may both employ more lifeguards and require more rescues, reversing the naive causal interpretation.

The association can be generalized to the state’s public swimming pools during the summer studied. It does not show that adding lifeguards causes more rescues, because higher-risk pools may both schedule more lifeguards and have more rescues.

- What response did the network test record? - Which pages were eligible for random selection? - Was any customer purchase behavior observed?

1. The random page sample supports conclusions about the store's current website pages under the test conditions. 2. Random assignment supports a causal comparison of image formats on page-load time. 3. Purchases and customer satisfaction were not measured. 4. A faster observed page-load time alone would not prove a business benefit.

The design can compare the formats causally for page-load time on the current website under the tested conditions. It cannot establish higher purchases or satisfaction because those outcomes were not measured.

- What was awarded by chance? - Is the lottery population every eligible household or only applicants? - Does the design by itself show the direction of an effect?

1. The lottery randomly assigns the offer of a voucher among applicants. 2. It therefore permits a causal comparison of receiving the offer, not necessarily actual voucher use, for lottery entrants. 3. Applicants were not randomly selected from all eligible households; people who did not apply are outside the study. 4. Random assignment alone does not establish that an offer improves housing stability.

The lottery supports a causal comparison of being offered a voucher among people who entered this city's lottery. It does not automatically generalize to eligible households that did not apply or to other cities.

- Trace the order from roster to sample to video groups. - Compare the roster with the district's full student population. - Connect sampling to generalization and assignment to a causal comparison.

1. Students are randomly sampled from the working-email list and then randomly assigned to one of the videos. 2. Students whose accounts are inactive cannot enter the sample, so the frame undercovers district students. 3. Random assignment supports a causal comparison of the videos on quiz score. 4. Random selection represents the working-email-list population, not automatically every student in the district. 5. The design itself does not prove that one video raises scores.

a) Random sampling: the SRS of \(200\) students from the working-email list. Random assignment: assigning sampled students to the two video types by chance. b) Undercoverage: students with inactive email accounts are excluded. c) The study can compare the videos causally for students represented by the list, but it does not automatically generalize to all district students.

- Could a household that never volunteers online enter each plan? - In which plans might people's existing habits affect whether they ask for the reminder? - A plan can have a fair treatment comparison without representing everyone. Look for that distinction.

1. Plan 1 draws households from the complete city list and uses a coin flip to send or not send a reminder. It can support both a citywide conclusion and a causal comparison, depending on the results. 2. Plan 2 randomly samples households, but those households choose whether to receive the reminder. It cannot isolate a causal effect of the reminder. 3. Plan 3 assigns sending or not sending the reminder by chance, but only among households that volunteer online. It can compare treatment effects among those volunteers, not reliably represent all city households. 4. Plan 4 uses volunteers who choose the reminder. It offers neither random-sample generalization nor a randomized causal comparison.

Plan 1 can address both goals. Plan 2 lacks random treatment assignment, so household differences may explain any effect. Plan 3 lacks a citywide random sample, so volunteers may not represent other households. Plan 4 lacks both random steps and supports neither broad conclusion by design alone.

- Which households could never enter a study that recruits only garden-store shoppers? - Might the people who pick an illustrated guide already differ from those who pick the text-only guide? - Consider whether your redesign fixes both problems or just one.

1. Randomly select households from a complete county household list rather than taking only households represented by garden-store shoppers. 2. Randomly assign the selected households to receive either the illustrated or text-only guide, then measure compost sorting after the same period. 3. Better recruitment alone would not separate the guides' effects, since households choosing a guide might already differ in composting habits.

Draw households at random from a complete county address list, then assign each selected household a guide by chance. Measure compost sorting after a month. A random sample alone would not fix self-selection into the guide types.

- What was assigned by chance? - What stress outcome was actually recorded, and what other outcome was not? - How were volunteers recruited into the study?

1. Random assignment supports a causal comparison of app use on reported stress among the participants. 2. The lower observed group mean alone is not proof of a true effect. 3. Self-reported stress does not necessarily measure physiological stress. 4. Volunteers were not randomly sampled from all adults, so the results do not automatically generalize to that population.

The study can investigate whether the app causes a change in reported stress for its volunteers, but an observed difference alone does not prove one. Reported stress may differ from physiological stress, and volunteers do not represent all adults through random sampling.

- What was assigned by chance: an offer or an actual bus trip? - Which later outcome did the clinic record for every participant? - Did the participants join through a random sample of all clinic patients?

1. Researchers randomly assigned an offer, not actual pass use. 2. The groups can therefore be compared to study whether being offered a pass affects missing the next appointment. 3. Patients decide whether to use a pass, so the separate effect of actually riding the bus is not isolated by random assignment. 4. These volunteers are not a random sample of all clinic patients.

It can study the effect of being offered a free pass on missing the next appointment, not the separate effect of using the bus. The findings concern these volunteers and do not automatically apply to all clinic patients.

- Where did the randomly sampled trout come from? - What did random assignment change for the fish? - Which details of time and environment are specific to this test?

1. Random sampling represents juvenile trout from the sampled lake. 2. Random assignment allows causal comparison of the tested temperatures under the laboratory conditions. 3. The results are tied to the two temperatures and \(30\)-day period. 4. Other lakes, adult trout, and natural field conditions may differ from the sampled and laboratory conditions.

The design supports a causal comparison of the two tested temperatures for juvenile trout from this lake in the laboratory. Two limits are the \(30\)-day duration and the fact that laboratory conditions may not match natural lake conditions.

- Identify the observational unit represented by each data point. - Keep the conclusion at the same level as the measured variables. - Ask whether any individual person's exposure and outcome were paired in the data.

1. Random selection of counties supports generalizing the observed county-level association to counties in the population represented by the sample. 2. The variables are county averages or rates, so the observational units are counties. 3. The study does not link individual residents' park access with their individual obesity status. 4. Therefore, it cannot conclude that a particular person with greater park access is less likely to have obesity, and it does not establish causation.

The study supports a population-level association among counties: counties with more park acreage per resident tend to have lower adult obesity rates. It does not support an individual-level claim about residents, and it does not establish that parks cause lower obesity rates.

- Is each resident assigned separately, or is one program chosen for a whole building? - Which units were randomly selected from the city? - Keep the level of the conclusion the same as the assigned unit.

1. Entire buildings, not residents, receive treatment assignments. 2. Random selection can represent the city's apartment buildings covered by the sampling frame. 3. Random assignment allows a causal comparison of building-wide programs on building-level participation rates. 4. It does not create individually randomized treatments for residents or establish a positive program effect by itself.

Buildings are the experimental units. The design can compare the two building-wide programs causally for apartment buildings covered by the city's sample. It does not support treating residents as independently assigned participants.

- Trace the process from invitation to actual participation. - Which step depends on the student's own decision? - What random step occurs after the volunteers agree to participate?

1. Random assignment supports a causal comparison of study tools among participating students. 2. The initial invitation list was random, but actual participation depended on each student's decision to consent. 3. Participants may differ from those who did not volunteer, so the final study group is not necessarily representative of all enrolled students. 4. The design alone does not show either tool improves scores.

The experiment can compare the tools causally among participating students. Because only invited students who consented were assigned, the study does not automatically generalize to everyone at the university.

- Which two people are compared within each pair? - Which diet assignment is made at random? - Where did the twin pairs come from, and does that represent all adults?

1. Random assignment within each twin pair allows a causal comparison of the diets for the participating twins. 2. Pairing controls for genetics and shared background, but the design alone does not show which diet lowers cholesterol. 3. The pairs were volunteers from a special registry and were not randomly sampled from all adults.

The study can compare the diets causally among the participating twins, using pairs to control shared characteristics. The volunteer twin sample does not support automatic generalization to all adults.

- Identify the condition required for a patient to enter the dataset. - Keep the conclusion within the group satisfying that condition. - Ask whether the excluded patients are necessary for studying the survival outcome.

1. The dataset is restricted to patients who survived at least \(30\,\text{days}\), so conclusions apply only to that survivor population. 2. Rehabilitation timing was observed rather than randomly assigned, so the study supports an association, not causation. 3. Patients who died within \(30\,\text{days}\) are excluded by design. 4. Because the outcome used to define inclusion removes all nonsurvivors, the study cannot estimate or compare \(30\)-day survival for rehabilitation groups.

Among \(30\)-day survivors, earlier rehabilitation is associated with shorter hospital stays. The study does not establish causation and cannot draw conclusions about \(30\)-day survival because nonsurvivors were excluded.

- What is the smallest unit to which the promotion is assigned separately? - Which dates and stores could enter the random sample? - Were individual shoppers assigned to treatments?

1. One store on one day receives a separate assignment, so a store-day is the experimental unit. 2. Random sampling can represent the store-days on the October–December schedule. 3. Random assignment permits causal comparison of promoting versus not promoting on daily sales totals. 4. It does not independently assign a treatment to each customer and does not by itself prove an increase in sales.

Experimental unit: a store-day. The design can compare the promotion causally for store-days on the October–December 2026 schedule. It does not prove the promotion raises sales or establish shopper-level treatment effects.

- Distinguish making groups similar from assigning exposure by chance. - Identify variables that may not have been included in the match. - Phrase the conclusion as an association when self-selection remains.

1. Matching can make the two groups more similar on the variables used for matching. 2. Meal-delivery use was still self-selected rather than randomly assigned. 3. Unmeasured differences, such as work schedule or interest in cooking, may remain confounded with service use. 4. The study supports an association between service use and less cooking time among the matched people, but not a causal effect.

Matching does not create random assignment. The study supports an association between meal-delivery use and less cooking time among the matched people, but it does not establish that the service caused the reduction.

- Was the analysis performed for the entire sample or separately by group? - Could an overall average hide different patterns in different groups? - What additional comparisons would be needed to discuss a subgroup?

1. Random sampling allows findings about the club's adult members, and random assignment allows a causal comparison of the plans. 2. An overall comparison does not show the size or direction of any effect within each age or sex subgroup. 3. Such claims need adequate data and comparisons for the subgroups themselves.

No. The design can study an average causal effect for adult club members overall, but a result for everyone combined does not establish the same effect for each subgroup or person.

- Compare the proportions of missing outcomes across the two assigned groups. - Ask whether the people who remain are still likely to be comparable. - Distinguish the originally assigned sample from the smaller group included in the final analysis.

1. The original random selection and random assignment would support population-level causal inference if outcome data were retained comparably across groups. 2. Large differential dropout can make the remaining treatment and control groups systematically different. 3. A comparison of completers may therefore reflect both the treatment and differences in who remained. 4. The clean causal conclusion for all sampled employees, and its generalization to all employees, is weakened unless the missing outcomes are addressed appropriately.

Differential attrition can destroy the comparability created by random assignment among the analyzed completers. A complete-case comparison may not support a causal conclusion for all sampled employees or a generalization to all employees without addressing the missing outcomes.