- Ask whether the researchers impose a condition on participants. - Distinguish recording an existing behavior from assigning a treatment.

1. The researchers only record existing behavior and outcomes. 2. They do not impose or assign a caffeine treatment. 3. Therefore, the study is observational.

It is an observational study because the researchers measure existing caffeine use rather than assigning a caffeine treatment.

- Treat recruitment and treatment assignment as two separate steps. - Ask where chance is used in each step.

1. The volunteers chose to enroll after seeing a public flyer, so they were not randomly selected from a larger population. 2. A random number generator determines which exercise routine each enrolled volunteer receives. 3. Therefore, the study uses random assignment but not random selection.

Random assignment only. The volunteers were not randomly selected, but their exercise routines were assigned by chance.

- What is the smallest unit that gets its own fertilizer assignment? - Can two seedlings in the same tray receive different assigned fertilizers? - Distinguish the measured seedlings from the units assigned a treatment.

1. The fertilizer treatment is assigned separately to each tray. 2. Seedlings within a tray do not receive independently assigned treatments. 3. Each tray is an experimental unit.

The trays are the experimental units.

- What do the researchers change or assign? - What do they measure afterward? - Include a unit for the numerical outcome.

1. The conditions assigned are drawing in silence and drawing with music. 2. The measured outcome is how many minutes each drawing takes to finish.

Treatments: silence and music. Response variable: time to finish the drawing, in minutes.

- Did the coach decide how long anyone should sleep? - Are the researchers changing a condition or only recording it? - What distinguishes an experiment from an observational study?

1. The coach records existing sleep habits and sprint times. 2. No sleep treatment is assigned to athletes. 3. This is an observational study.

An observational study, because the coach does not assign a sleep treatment.

- Could students choosing one method already differ from those choosing the other? - Would those differences affect vocabulary scores? - What does assigning by chance do about this problem?

1. Students who prefer different methods may also differ in motivation or prior skill. 2. Self-selection can mix those differences with the method being compared. 3. Random assignment helps make groups comparable before the treatments.

Random assignment helps prevent students' existing differences and preferences from being tied to one study method.

- What would attendance look like without the new message? - Which group provides that reference? - Why is a comparison useful?

1. The usual-message group is the control group. 2. It provides a comparison for attendance under the new message.

It is the control group and provides a comparison for judging the new reminder message.

- For each plan, count the different players, not just the serve measurements. - Can repeated serves from one player represent the differences between players? - What is the unit receiving the grip in each plan?

1. The first coach measured one player five times; these are repeated measurements of the same experimental unit. 2. The second coach used five different players, so the grip was tested on several units. 3. Replication across players comes from the second plan, not merely from taking more measurements of one player.

The second coach has replication across players. The first coach has repeated serve measurements from one player, not replication across players.

- Use classrooms that are as similar as possible before treatment. - Randomly assign the coating rather than choosing the hottest rooms. - Measure the same temperature outcome on the same schedule.

1. Randomly assign similar portable classrooms to receive the reflective coating or remain uncoated. 2. Keep thermostat settings, measurement times, occupancy schedules, and window use consistent. 3. Record interior temperature with the same calibrated sensors over comparable weather periods. 4. Compare average temperatures to estimate the causal effect of the coating under the study conditions.

Randomly assign similar classrooms to coated and uncoated groups, hold operating conditions constant, and compare their measured interior temperatures.

- Find what receives a treatment. - Separate the imposed condition from the measured outcome. - Determine whether units were grouped before random assignment. - Notice which other conditions were held constant.

1. The experimental units are the thirty-six seedlings. 2. The treatments are the three fertilizer conditions. 3. The response variable is final dry mass. 4. Treatments are assigned completely at random, so this is a completely randomized design. 5. Keeping environmental conditions constant is direct control.

Experimental units: the thirty-six seedlings. Treatments: the three fertilizers. Response variable: final dry mass. Design: a completely randomized design, with light, water, and temperature directly controlled.

- Count how many experimental units receive each treatment. - Ask whether natural variation can be separated from the treatment effect. - Add enough units and a fair assignment process to the redesign.

1. Each treatment is applied to only one experimental unit. 2. Any observed difference could be caused by natural plant-to-plant variation rather than the soil mixture. 3. The experiment should assign many similar plants to each mixture, using random assignment and otherwise comparable growing conditions.

There is no replication within treatments because only one plant receives each mixture. Use multiple plants per mixture, randomly assign plants to mixtures, and keep other growing conditions comparable.

- Identify the factors and count the levels of each. - A treatment is not a single factor level when more than one factor is imposed. - List a few combinations to see the structure.

1. Temperature has \(2\) levels and baking time has \(3\) levels. 2. Treatments are combinations of factor levels. 3. The number of treatments is \(2\cdot3=6\). 4. One treatment is one specific temperature paired with one specific baking time.

There are \(6\) treatments. Each treatment is one combination of an oven temperature and a baking time.

- Determine whether participants receive one treatment or both. - Identify what serves as the match. - Think about a major source of variability removed by comparing within a person.

1. Each participant receives both treatments, with randomized order. 2. This is a matched pairs design in which each participant serves as their own control. 3. Person-to-person differences in reading speed are controlled because the treatment comparison is made within each participant.

It is a matched pairs design. Having each participant use both settings controls individual differences in baseline reading speed and can make the comparison more precise.

- Identify conditions that are not treatments but could affect the response. - Notice that these conditions are kept the same for every unit. - Explain how that strengthens the treatment comparison.

1. Holding batch, exposure, and development conditions constant is direct control. 2. These conditions are possible extraneous sources of variation in image quality. 3. Direct control prevents systematic differences in those conditions from competing with the temperature explanation.

The principle is direct control. Keeping batch, exposure, and development conditions constant reduces extraneous variation and helps isolate the effect of storage temperature.

- List every group that knows the assignments. - Use the masking label based on both participants and interacting researchers. - Consider how informed technicians could influence outcomes.

1. Participants are unaware of treatment assignments. 2. Interacting technicians know the assignments. 3. Therefore, the study is single-blind, or single-masked. 4. Technicians may communicate expectations or assess outcomes differently, so researcher-related bias can remain.

The study is single-blind because only participants are unaware of the assignments. Technician behavior or evaluation may still differ by treatment and bias the responses.

- Give every participant a unique label. - Randomly choose the required number for one treatment without repeats. - Use the unselected participants for the other treatment. - Verify that the group sizes are equal.

1. Label participants \(01\) through \(48\). 2. Use a random number generator to select \(24\) distinct labels without replacement. 3. Assign those participants to Treatment A and assign the remaining \(24\) participants to Treatment B. 4. This gives every participant a random treatment assignment and equal group sizes.

Label the participants \(01\) through \(48\), randomly select \(24\) distinct labels for Treatment A, and assign the other \(24\) participants to Treatment B.

- Divide the specimens equally among three treatments. - How can the lab assign specimens without favoring a coating? - Which measured outcome will show how well the coatings resist wear?

1. Label all \(75\) specimens. 2. Randomly assign \(25\) different specimens to each coating. 3. Apply the coatings and run the same wear test under controlled conditions. 4. Record mass lost from each specimen and compare the groups.

Randomly assign \(25\) specimens to each coating, keep test conditions the same, and compare the mass lost in the standard wear test.

- Ask which characteristic is measured before treatment. - Compare how strongly each proposed characteristic could predict the response. - Choose the variable that would make units within a block more similar on the outcome.

1. A useful blocking variable is measured before treatment and is strongly related to the response. 2. Baseline reading score is likely related to reading improvement or final reading performance. 3. Eye color has no plausible instructional relationship to the response. 4. Participants should be grouped by similar baseline reading scores, then randomly assigned to interventions within each block.

Block by baseline reading score because it is a pre-treatment variable likely related to the response. Eye color is not a useful source of response variation for this study.

- Check every source of information a participant sees, not only the tablets. - Ask whether any label reveals the assigned condition. - Replace descriptive labels with a code that is unavailable to people who should be masked.

1. The bottle labels reveal treatment assignment to participants, so identical tablet appearance does not mask them. 2. Knowing the assignment can change reported symptoms or behavior through expectations. 3. Bottles should use neutral coded labels prepared by someone not involved in outcome measurement. 4. The treatment code should remain concealed from participants and relevant research staff until data collection is complete.

Masking fails because the bottle labels disclose the assignment. Use identical neutrally coded containers and keep the code hidden from participants and outcome assessors until the study is finished.

- Identify the grouping that exists before fertilizer is assigned. - Decide whether random assignment is carried out once across all seedlings or separately inside those groups. - The experimental unit is the smallest unit that receives its own fertilizer assignment.

1. The seedlings are first separated by greenhouse bench before treatment assignment. 2. Fertilizer assignment is randomized separately within each bench, so treatment comparisons are made within the bench groups. 3. Therefore, the design is a randomized block design rather than one completely randomized assignment across all seedlings together. 4. The blocks are the North and South benches, and the experimental units are the individual seedlings.

It is a randomized block design because fertilizer is randomized separately within each bench rather than across all seedlings together. The blocks are the North and South benches, and the experimental units are the individual seedlings.

- What receives a soil treatment before anything is measured? - Could leaves from the same pot have different assigned soil treatments? - Use the branches to count the units that received assignments.

1. The treatment is assigned separately to pots, not to the leaves measured later. 2. The \(12\) compost pots and \(12\) regular-soil pots make \(24\) experimental units. 3. Four leaves from one pot are repeated measurements within that pot; they are not four independently assigned units.

No. There are \(24\) experimental units—the pots. Measuring several leaves from a pot does not create more independently assigned units.

- Could starting math skill affect the final quiz even if both practice methods work equally well? - Look at which students are compared after the first split. - Think about a difference between students that the center is trying not to mistake for a practice effect.

1. Students with different starting math skills might get different final scores even if practice style has no effect. 2. Grouping by starting score and then assigning practice at random within each group helps compare students with similar starting skills. 3. Random assignment alone is helpful, but blocking may further reduce differences between the groups unrelated to practice style.

Grouping by starting math scores makes the practice groups easier to compare fairly: students with similar prior skill receive different methods by chance. Random assignment and blocking serve different purposes.

- Compare the two times in each student's row. - Could students differ in how quickly they normally solve puzzles? - If one puzzle is harder, what would happen if it were always used with music?

1. Priya took \(12\) minutes in silence and \(10\) with music, so she was \(2\) minutes faster with music. Jordan was slower with music, and Amir had equal times. 2. Each student's two times form a pair. Comparing within that pair keeps differences in students' usual puzzle-solving speeds from being mixed with the music comparison. 3. Pairing different students would lose that control. Randomizing which puzzle gets music prevents puzzle difficulty from always favoring one condition.

Priya was \(2\) minutes faster with music. Comparing each student with themselves avoids mixing different students' usual speeds. Randomizing which puzzle gets music prevents a harder puzzle from always being linked to one condition.

- What else changes between Monday and Friday besides the recipe? - How could both recipes be offered on the same day? - What part of the new plan should be decided by chance?

1. The recipe is linked to the weekday, so differences between Monday and Friday customers could affect the ratings. 2. Offer both soups on each day and randomly assign consenting students to receive one of them. 3. Compare ratings within or across the two days, keeping the assigned recipe separate from weekday.

Recipe and weekday are confounded. On both Monday and Friday, randomly assign consenting students to lentil or tomato soup so both soups are served each day.

- Compare only treatments that meet safety standards. - Randomize the routine rather than allowing teams to choose. - Use a rate that adjusts for unequal time at risk.

1. Randomly assign teams or athletes to approved Routine A or approved Routine B; do not assign an unsafe no-warm-up condition. 2. Keep training schedules, injury definitions, and reporting procedures consistent. 3. Record muscle strains per \(1000\) athlete-hours of participation so different exposure amounts are comparable. 4. Compare injury rates to estimate the causal difference between the two approved routines.

Randomly assign participants to the two approved routines and compare muscle-strain rates per \(1000\) athlete-hours using the same injury-reporting rules.

- Identify what happens before the treatment assignment. - Check whether random assignment occurs within groups. - Think about why prior performance could affect the response.

1. Students are grouped by a variable expected to affect the final-exam score. 2. Treatments are randomly assigned within each group. 3. The design is a randomized block design. Blocking separates variation associated with previous performance from the treatment comparison.

It is a randomized block design. Blocking by previous test score helps control variation in final-exam performance related to prior achievement, allowing a more precise comparison of the two review formats.

- Notice how many treatments each participant receives. - Identify what is paired in the design. - Consider how receiving one condition first could affect the response.

1. Each participant receives both treatments, so each participant serves as their own control. 2. This is a matched pairs design. 3. Randomizing the order helps prevent an order effect from being confounded with the label treatment.

It is a matched pairs design. Randomizing presentation order helps keep fatigue, adaptation, or first-taste effects from systematically favoring one label condition.

- Determine whether one treatment has an active ingredient. - Identify exactly who does and does not know the assignments. - Connect each design feature to a possible source of altered responses or ratings.

1. The inactive cream is the placebo control treatment. 2. Neither participants nor interacting evaluators know the assignments, so the experiment is double-blind, or double-masked. 3. The placebo helps separate the treatment effect from expectations, while double masking reduces response and evaluator bias.

The inactive cream is the placebo control, and the study is double-blind. The placebo controls for expectation effects, and double masking reduces bias from participants' and clinicians' knowledge of treatment assignments.

- Identify a source of response variation unrelated to the treatment. - Group units that are similar with respect to that source. - Make sure every treatment appears within every group.

1. Bench location may affect plant growth because light conditions differ. 2. Form blocks based on location, such as window and interior benches. 3. Randomly assign all three watering schedules within each block. 4. Blocking separates variation due to location from variation used to compare watering schedules.

Create blocks based on bench location and randomly assign each watering schedule to plants within every block. This controls location-related light variation and makes the watering comparison more precise.

- Identify how experimental units are grouped before assignment. - Check the number of treatments. - Think about road characteristics that could affect the response apart from sealant type.

1. The experiment has two treatments and road sections are paired on relevant characteristics. 2. Treatments are randomly assigned within each pair. 3. This is a matched pairs design. 4. Matching controls variation in traffic, pavement age, or condition that could affect wear.

It is a matched pairs design. Pairing similar road sections controls important preexisting differences, so wear can be compared within pairs with less extraneous variation.

- Use the baseline characteristic that is expected to affect improvement. - Ensure every treatment appears in every block. - Randomize only after the blocks are formed. - Measure the same response for all patients.

1. Form three blocks based on mild, moderate, and severe baseline limitation. 2. Within each block, randomly assign patients to all three exercise programs. 3. Measure the same mobility response after treatment. 4. This design controls baseline severity while allowing program comparisons within severity levels.

Create blocks for mild, moderate, and severe baseline limitation. Randomly assign patients within each block to each of the three programs, then compare the common mobility response while accounting for the blocks.

- Distinguish a long-run property from a guarantee for one assignment. - Ask whether the assignment mechanism used chance correctly. - Explain why chance groups can still differ somewhat.

1. Random assignment does not guarantee identical groups on every observed characteristic. 2. Chance imbalances can occur in a particular randomization. 3. Random assignment uses chance rather than researcher choice and makes treatment groups comparable on average across repeated random assignments. 4. It reduces systematic confounding and supports causal inference when the experiment is otherwise well designed.

A baseline difference can occur by chance, so it does not prove random assignment failed. Random assignment prevents systematic selection into treatments and balances extraneous variables in the long run, not perfectly in every experiment.

- Identify a preexisting group that may affect the response. - Ensure both treatments occur within every group. - Randomize after forming the groups.

1. Treat each shift as a block because productivity and fatigue may differ by shift. 2. Within each shift, randomly assign workers to the two break schedules. 3. Measure the same productivity or fatigue response for all workers. 4. Compare schedules within shifts and combine the evidence across blocks.

Form blocks by day, evening, and night shift. Randomly assign workers within each shift to each break schedule, then compare the common response while accounting for shift.

- Identify the decision the company actually needs to make. - Compare the new option with what would normally occur. - Consider both relevance and ethics when choosing a control.

1. The current checklist represents the existing standard practice. 2. Comparing the new checklist with current practice isolates improvement over what workers would otherwise receive. 3. A no-checklist group changes both checklist content and the presence of any checklist, making the comparison less relevant and possibly unethical.

The current checklist is an appropriate active control because it represents standard practice. A no-checklist condition would not answer whether the new checklist improves on current practice and could reduce safety.

- List who knows the treatment assignment. - Identify who records the response variable. - Connect masking to the judgment most directly affected.

1. The outcome evaluator is unaware of treatment assignments. 2. Participants and treating therapists are not masked. 3. Masking the evaluator reduces the chance that knowledge of treatment influences measurement or scoring. 4. It does not eliminate expectation effects among participants or therapists.

The outcome evaluator is masked. This reduces evaluator measurement bias, although participant and therapist expectations can still affect the response.

- Identify what the new method is compared against. - Explain why treatment assignment should use chance. - Explain why one plot per method would be inadequate.

1. Comparison with the standard method provides a control benchmark. 2. Random assignment reduces systematic differences between treatment groups. 3. Replication across many plots reveals treatment effects despite natural plot-to-plot variation. 4. Together these principles support a causal comparison.

The standard method supplies a comparison group, random assignment reduces confounding, and replication across many plots helps separate treatment effects from natural variation.

- Identify a pre-treatment variable strongly connected to the outcome. - Decide whether grouping similar units can reduce unexplained variation. - Ensure random assignment remains within each group.

1. Baseline reading skill is an important source of extraneous response variation. 2. Forming blocks of students with similar baseline skill separates that variation. 3. Randomly assign both study methods within every block. 4. A randomized block design should give a more precise comparison than complete randomization.

Use a randomized block design based on baseline reading skill, then randomly assign both study methods within each block. Blocking controls a strong predictor of the response.

- Identify the real-world alternative to adopting the new lesson. - Ask what a no-instruction group would change besides lesson type. - Consider whether withholding the usual service is appropriate.

1. The relevant decision is whether the new lesson improves on current instruction. 2. The current lesson provides a fair standard-practice comparison. 3. No instruction would combine the effect of lesson content with the effect of receiving instruction at all. 4. Withholding instruction may also be educationally inappropriate.

Use the current lesson as the active control. It directly tests whether the new lesson improves on standard instruction and avoids the confounding and ethical problems of withholding instruction.

- Decide whether chance determines each patient's treatment. - Look for a characteristic that may change across the arrival sequence. - Ask whether that characteristic could also affect the response to the reminder.

1. Alternating assignments follow a predictable rule and do not use chance, so the treatment assignment is not random. 2. Arrival order may be related to appointment time, work schedule, transportation access, or patient type. 3. If one script is systematically given to patients arriving in different time slots or circumstances, the script effect cannot be separated from the arrival-order difference. 4. A valid redesign would use a random mechanism to assign each patient, while keeping treatment counts reasonably balanced.

Alternation is not random because the next assignment is predetermined. Arrival order could be associated with patient characteristics or appointment times, making those factors confounded with the script. The clinic should assign scripts with a random mechanism.

- Read the three mean decreases from the bar heights. - For each comparison, subtract the appropriate two group means. - Could an inactive pill change anything besides what someone expects?

1. The observed mean decrease is \(3\) points for inactive pill and \(1\) point for no pill. Their difference is \(3-1=2\) points. 2. The observed mean decrease is \(8\) points for medication and \(3\) points for inactive pill. Their difference is \(8-3=5\) points. 3. Receiving an inactive pill can also change attention, care, or routine, so the first contrast is not a pure measure of expectations.

(a) \(2\) points. (b) \(5\) points. (c) The first difference could include effects of taking a pill or receiving attention, not just expectations.

- Separate randomness from balance. - Ask whether a chance process can produce unequal outcomes in a finite number of trials. - Use a random procedure with a fixed number of labels for each treatment when exact balance is required.

1. Independent fair coin flips are a valid random-assignment mechanism. 2. Random assignment does not guarantee equal group sizes, so a split of \(23\) to \(17\) can occur by chance. 3. To guarantee equal sizes, prepare \(20\) A labels and \(20\) B labels, shuffle them thoroughly, and assign one label to each participant without replacement.

The assignment is random despite the unequal group sizes. Equal sizes are not guaranteed by independent coin flips. To force two groups of \(20\), randomly distribute \(20\) A labels and \(20\) B labels without replacement.

- Count the number of experimental units in each group. - Check whether random assignment occurs inside the groups. - Compare the group size with the defining structure of matched pairs.

1. The groups are formed using a pre-treatment variable related to the response. 2. Treatments are randomly assigned within each group, so the study uses a randomized block design. 3. It is not a matched-pairs design because each block contains four participants rather than a pair, and two units receive each treatment.

This is a randomized block design with baseline blood pressure as the blocking variable. It is not matched pairs because each block contains four participants, not two matched units.

- Identify when the delayed group has not yet received the treatment. - Choose a measurement time when the treatment conditions still differ. - Use random assignment to interpret the group difference at that time.

1. During the first eight weeks, the wait-list group represents what happens without immediate access to the workshop. 2. Random assignment makes the immediate and delayed groups comparable at the start. 3. The response should be compared between groups at the end of eight weeks, before the wait-list group receives the workshop. 4. That comparison estimates the causal effect of immediate workshop access over the eight-week period.

The wait-list is the control group during the first eight weeks because it does not yet receive the treatment. Compare the groups' outcomes at eight weeks, before the wait-list begins the workshop, to estimate the effect of immediate access.

- Separate the quality of the matching from the treatment-assignment rule. - Ask whether chance determines which member of a pair receives each treatment. - Identify the characteristic that is linked perfectly to treatment in the current plan.

1. The pairing is useful because machines in each pair have similar workload. 2. Treatment assignment within a pair is not random; machine age determines the lubricant. 3. Any lubricant difference may be confounded with age because A always goes to the older machine. 4. Within each pair, use a random mechanism to decide which machine receives A and assign B to the other machine.

The design lacks random assignment within pairs, so lubricant is confounded with machine age. Randomly assign Lubricant A to one machine in each pair and Lubricant B to the other.

- List what differs between groups besides the assigned program. - Ask whether that extra difference could affect the response. - Hold the interaction constant or prevent the tester from knowing treatment assignments.

1. Coach encouragement varies with treatment assignment and can affect test performance. 2. The observed response difference may reflect both the fitness program and unequal encouragement. 3. Use a scripted testing protocol with the same interaction for every participant and, when possible, mask testing staff to treatment assignment. 4. Automated or independent outcome measurement can further reduce the influence of coach expectations.

Unequal encouragement is an extraneous treatment-linked difference that can inflate the apparent effect of the new program. Standardize the testing interaction and mask or separate outcome assessors from treatment information.

- Identify the groups formed before the treatments are assigned. - Determine the smallest objects that independently receive an irrigation treatment. - Multiply the number of plots receiving one treatment in each soil zone by the number of soil zones.

1. The plots are grouped by soil zone before treatments are assigned, so the \(4\) soil zones are the blocks. 2. The irrigation treatments are assigned to individual plots, so the \(24\) plots are the experimental units. 3. Each treatment is assigned to \(2\) plots in each of the \(4\) soil zones, so each treatment has \(4\cdot 2=8\) replications overall.

Blocks: the \(4\) soil zones. Experimental units: the \(24\) individual plots. Replication: \(2\) plots per treatment in each soil zone, or \(8\) plots per treatment overall.

- What else might change between one week and the next? - What routes could keep the old signs while others get new signs? - Why should both conditions be measured during the same week?

1. The two weeks may differ in visitors, staffing, or crowding; the sign change is mixed with changes over time. 2. Randomly assign several comparable exhibit routes to keep the old signs or get the new signs. 3. During the same week, measure visitor search times on the routes using the same rules. 4. Compare groups of routes, not two time periods.

The before-and-after comparison confounds signs with other changes between weeks. Randomly assign comparable exhibit routes to old or new signs and compare visitor search times during the same week.

- Can the percentage difference change as more shoppers are included? - Picture two equally good checkout pages. Could their observed percentages still differ along the way? - Think about how the choice of the final group of shoppers might affect the reported result.

1. Even if the pages work equally well, random variation can temporarily put the new page ahead. 2. Checking repeatedly and stopping only at a large lead favors reporting such a temporary difference. The study might also continue indefinitely if that lead never appears. 3. Decide a sample size or end date before seeing purchase results and stop as planned, whichever page is ahead.

A chance lead could trigger the stop and make the new page seem better than it is. Choose the sample size or end date before collecting results, and use that stopping point regardless of which page leads.

- Trace one branch for each combination of factor levels. - Make sure chance determines which treatment combination each volunteer receives. - Ask whether the effect of one factor should be compared across the two levels of the other factor.

1. The two factors are music status and notification status, each with two levels. 2. The four treatments are music with notifications, music without notifications, no music with notifications, and no music without notifications. 3. Randomly assign \(40\) volunteers to each treatment, give everyone comparable proofreading tasks under controlled conditions, and compare accuracy across the four groups. 4. To assess whether the effect of music depends on notification status, compare the music-versus-no-music difference in mean accuracy when notifications are on with the corresponding difference when notifications are off. Different music effects across the two notification conditions indicate an interaction.

Randomly assign \(40\) volunteers to each treatment: music with notifications, music without notifications, no music with notifications, and no music without notifications. Keep the proofreading task and other conditions the same. To assess whether the effect of music depends on notification status, compare the music-versus-no-music difference in mean accuracy with notifications on to the corresponding difference with notifications off. If those effects differ, the factors interact.

- Ask who can infer the assigned condition before the outcome is measured. - Consider how expectations could change behavior as well as reports. - Think about which sensory features a convincing comparison drink must match.

1. Athletes’ expectations can influence effort or self-reported symptoms because the treatment is recognizable. 2. Coaches may treat or encourage athletes differently when they know the assignment. 3. A better placebo should match the treatment drink in appearance, taste, and packaging but omit the active ingredient. 4. Codes can conceal assignments from athletes, coaches, and outcome assessors when feasible.

The recognizable drinks allow participant expectation and coach behavior to differ by treatment. Use a placebo matched in taste, appearance, and packaging, and conceal the assignment from participants, coaches, and assessors when possible.

- Ask whether anyone can know the next assignment before a patient enters the study. - Consider how that knowledge could affect which patient receives which condition. - Think about a procedure that prevents prediction and choice of assignments.

1. Staff can learn upcoming assignments before enrolling a patient. 2. Choosing envelopes based on patient characteristics can create systematic differences between treatment groups, defeating random assignment. 3. Use opaque, sealed, sequentially numbered envelopes or a centralized computer assignment that cannot be known or changed in advance.

The assignment can be manipulated because staff can see and choose the treatment in advance, so the groups may not be randomly formed. Use concealed, tamper-resistant sequential assignments or a centralized randomization system.

- List everything that changes at the same time as the sign. - Identify an alternative explanation for the sales difference. - Redesign the study so each sign is used under comparable conditions. - Include random assignment in the improved plan.

1. Sign type is completely linked to day and time. 2. Customer traffic and buying patterns may differ between weekday mornings and weekend afternoons. 3. Therefore, sign effect cannot be separated from day-and-time effects. 4. An improved design randomly assigns the signs across comparable time periods, with both signs used in each relevant block such as day type.

The sign treatment is confounded with day and time, so higher sales cannot be attributed to Sign B. One improvement is to block comparable selling periods by day type and randomly assign both signs within each block.

- What does the inactive-pill group receive that the no-pill group does not? - What is kept more similar when active treatment is compared with an identical-looking inactive pill? - Why choose group assignments by chance?

1. The inactive-pill and no-pill groups differ in receiving an inactive pill, which can involve expectations and the act of taking a pill. 2. Comparing active treatment with an identical-looking inactive pill helps isolate the added effect of the active ingredient. 3. Random assignment makes the group comparisons fairer, but measured differences can still contain chance variation.

The inactive-pill versus no-pill comparison shows the difference associated with receiving an inactive pill, including expectations and other features of taking a pill. Active treatment versus inactive pill compares the active ingredient while keeping the pill experience similar. Random assignment supports these comparisons.

- Evaluate the risk imposed by the proposed treatment. - Replace assigned exposure with naturally occurring variation. - Identify additional variables that should be recorded. - Adjust the type of conclusion to match the new design.

1. Deliberately imposing a potentially harmful exposure violates ethical standards. 2. A randomized experiment should not be conducted. 3. Researchers could compare naturally occurring exposure histories and hearing outcomes while measuring plausible confounding variables. 4. The observational study could identify association but would have weaker causal evidence.

Random assignment is unethical because the treatment could cause permanent harm. An ethical alternative is to observe people with different naturally occurring exposure levels, record hearing outcomes and potential confounders, and interpret the result as association rather than definitive causation.

- List every condition that changes with the program label. - Ask whether the proposed randomization changes those linked conditions. - Redesign so the treatment varies within tutor and time categories.

1. Program is completely linked with time of day and tutor. 2. Any difference in outcomes could be caused by the program, tutor, time, or a combination. 3. Randomizing students between the existing sessions would randomize participants but would not separate program from tutor and time. 4. Both programs must be used across tutors and times, with treatment assignment randomized within appropriate blocks.

Program, tutor, and time of day are confounded. Simply randomizing students into the two fixed sessions does not separate those effects. Use both programs with each tutor and time block, with random assignment within blocks.

- Identify the smallest unit independently assigned a treatment. - Distinguish repeated measurements from independent treatment assignments. - Count treatment replications at the assignment level.

1. The treatment is assigned to whole aquariums, not independently to individual fish. 2. Therefore, the aquariums are the experimental units. 3. Measuring ten fish per aquarium gives repeated measurements within each unit, not replication of treatment assignment. 4. With one aquarium per treatment, treatment is confounded with aquarium and there is no true replication.

The experimental units are the two aquariums because treatment is assigned at the aquarium level. The fish are subsamples, so there is only one experimental unit per treatment and no valid replication.

- Ask whether the first treatment can affect the later response. - Consider what could separate the two treatment periods. - Keep random order, but add a feature that addresses lingering effects.

1. A lingering effect from the first beverage can influence the response during the second period. 2. This is a carryover effect that can confound the second-period treatment response. 3. Add a sufficiently long washout period between treatments and keep the order randomized. 4. If washout is not possible, use separate randomly assigned groups instead of a crossover.

The study is threatened by carryover from the first beverage. Include an adequate washout period between treatment weeks, or use a parallel completely randomized design if carryover cannot be removed.

- Identify the smallest unit that can receive a different treatment independently. - Separate the assignment level from the measurement level. - Consider dependence among people sharing one treatment setting.

1. Treatment is assigned to work teams. 2. Therefore, teams are the experimental units. 3. Employees within the same team share one assignment and may have correlated outcomes. 4. Individual employee measurements are subsamples within experimental units, not independent replications of assignment.

The work teams are the experimental units. Employees within a team receive the same treatment, so their measurements do not represent separate random assignments or independent treatment replications.

- Identify a manufacturing source of preexisting variation. - Ensure both treatments occur within every level of that source. - Explain what would be inseparable if whole batches received only one treatment.

1. Production batch may affect bond strength, so batches should be blocks. 2. Within each batch, randomly assign components to both adhesives. 3. This permits adhesive comparisons within every batch. 4. Assigning one adhesive to each whole batch would confound adhesive with batch.

Treat each production batch as a block and randomly assign both adhesives to components within every batch. This controls batch variation and avoids confounding adhesive with batch.

- Find the smallest objects that receive treatments independently. - Look for pairs created to control a shared source of variation. - Check whether both treatments occur within every pair.

1. A treatment is applied separately to each test section, so the experimental units are the \(36\) panel sections. 2. The two sections cut from the same original panel form a block because they share the panel's material and preparation. 3. Each block contains two units and both treatments, with the assignment randomized within the block, so the design is matched pairs.

The experimental units are the \(36\) panel sections. The \(18\) original panels are the blocks, with the two sections from each panel forming a pair. The experiment uses a matched-pairs design.

- List every feature that changes when the treatment changes. - Ask whether the response difference could be attributed to more than one changing feature. - Revise the plan so the extra feature is held constant or represented under both treatments.

1. Treatment method is completely linked to technician, so any difference in response could be caused by the method, the technician, or both. 2. Blocking by initial hardness does not remove this confounding because technician still varies only with treatment. 3. A correction is to have each technician apply both methods, with treatment assignments randomized within technician and hardness blocks, or to use one standardized automated application process for both methods.

Technician is confounded with treatment because each technician applies only one method. The study should arrange for each technician to apply both methods under randomized assignments, or keep the applicator identical for both treatments.

- Determine when blocks must be formed relative to treatment assignment. - Ask whether the grouping variable could already have been affected by treatment. - Check whether treatments were actually randomized within the proposed groups.

1. Blocks must be formed before treatment assignment using a pre-treatment variable related to the response. 2. Grouping units by the response after the experiment uses outcome information that treatment may already have affected. 3. The original treatment assignments were not randomized within those post hoc groups, so the study did not use a randomized block design. 4. A future experiment could block on an appropriate baseline measurement and then randomize treatments within each block.

The proposed groups are not valid blocks because they are formed after treatment using the response itself. A randomized block design requires blocks based on pre-treatment characteristics and random assignment within each block before outcomes are observed.

- Decide which condition can realistically be held identical for all units. - Identify the pre-treatment characteristic that cannot be equalized but is related to the response. - Use random assignment after forming groups that are similar on that characteristic.

1. Room noise can be held at the same level for all participants, so it should be handled by direct control. 2. Baseline typing skill cannot be made identical, but it is related to typing speed, so participants should be grouped into skill blocks. 3. Within each block, participants should be randomly assigned to the two keyboard layouts. 4. This separates variation due to baseline skill while preventing room noise from varying across experimental units.

Keep room noise constant by direct control. Form blocks of participants with similar baseline typing skill, then randomly assign the two keyboard layouts within each block.

- Ask whether each plot receives only its assigned condition. - Consider how a neighboring plot could alter the measured response. - Look for a layout change that prevents treatments from crossing between units.

1. Runoff allows the treatment assigned to one plot to affect the response of a neighboring plot. 2. The control and treatment conditions are no longer kept separate, so the observed difference may be diluted or distorted. 3. The design can be improved by adding untreated buffer space between plots, using physical barriers, or randomizing treatments to larger separated plots that reduce spillover.

Fertilizer spillover causes interference between experimental units, so one plot's response may depend on neighboring treatments. Use buffer zones, barriers, or sufficiently separated treatment plots to keep the treatments isolated.

- Follow the sequence from slot labeling to participant choice. - Ask who ultimately determines which treatment a participant receives. - Separate treatment assignment from scheduling preference in the redesign.

1. Time slots were randomized, but participants selected their own slots. 2. Because treatment is tied to slot, participant preferences determine treatment assignment indirectly. 3. Morning and evening participants may differ in work schedules, sleep patterns, or other characteristics related to the response, creating potential confounding. 4. The researcher should first enroll participants and then randomly assign each participant to a treatment, while scheduling treatment delivery so both treatments occur across relevant times.

Participants were not randomly assigned because choosing a time slot also determined treatment. Randomly assign enrolled participants to treatments directly, and distribute both treatments across appointment times so time is not confounded with treatment.

- Recall why each block should contain every treatment being compared. - Ask what comparison remains possible when a block contains only one treatment. - Identify which block characteristic becomes linked to the treatment.

1. Blocking by hospital is intended to compare treatments within the same hospital while controlling hospital-to-hospital differences. 2. With no Treatment B patients at the affected hospital, there is no within-hospital treatment comparison there. 3. Treatment A is completely linked to that hospital's patients, so the hospital effect and treatment contribution cannot be separated within that block. 4. The trial no longer follows the intended randomized block design for that hospital, and the error must be addressed in analysis and reporting rather than treated as a valid block.

The affected hospital provides no within-block comparison because it contains only Treatment A. Hospital characteristics and Treatment A are inseparable there, so that block does not implement the planned randomized block design.

- Which features differ between the assigned groups? - Is there any group that gets only new software or only training? - Does a higher observed group mean by itself rule out chance variation?

1. Random assignment supports comparing the combined new package with the current system as a possible cause of productivity differences. 2. Software and training change together, so the experiment cannot identify their separate effects. 3. To study separate contributions, compare combinations of current/new software and with/without training. 4. Higher observed productivity in one group alone does not establish that the true effect is positive rather than a chance difference.

The design allows a causal comparison of the combined software-and-training package with the current system, but the observed increase could include chance variation. Separate software-only and training-only groups would be needed to compare the two components.

- Is the meaning of “low enough” fixed before the trial begins? - Could knowing someone's assigned program affect a judgment call? - When should enrollment criteria be settled to protect randomization?

1. Without a fixed rule, staff may judge similar initial test scores differently. 2. Knowing assigned programs could affect those judgments and change which applicants remain in each group. 3. That selection after assignment can damage the comparability established by randomization. 4. Define objective eligibility rules before assigning programs, then apply them without reference to assignment.

Staff could use different exclusion judgments in the two groups because the cutoff is not fixed and assignments are known. Set and apply objective eligibility rules before random assignment or apply a preplanned, assignment-blind rule.

- List every experience that differs between the two groups. - Identify a difference other than the intended program content. - Design a comparison group that matches time and attention while omitting the active content.

1. The treatment group receives both the stress-management content and regular group attention. 2. The control group receives neither component, so the effects of content and attention cannot be separated. 3. An attention-control group could meet for the same amount of time with the same group format but discuss neutral topics unrelated to stress management. 4. Comparing the treatment group with that control better isolates the effect of the program content.

The lower stress could result partly from regular attention or group contact rather than the specific program. Use a control group that meets equally often in the same format for neutral activities, without the stress-management content.

- Make the visible treatment experience the same for both groups. - Ensure each participant receives one active treatment and one inactive counterpart. - Keep the active component hidden through coded, matching delivery forms.

1. Every participant should receive both a pill and an injection. 2. One group receives the active oral medication and a placebo injection. 3. The other group receives a placebo pill and the active injected medication. 4. The active and placebo versions of each delivery form should look and feel as similar as safely possible, and treatment codes should be concealed from participants and outcome assessors.

Give every participant both delivery forms. Group 1 receives the active pill plus a placebo injection; Group 2 receives a placebo pill plus the active injection. Use concealed codes so participants and assessors do not know which component is active.

- Identify which decisions remain subjective after data collection. - Ask whether knowing the preferred group could influence those decisions. - Move decisions earlier or hide treatment identities until the planned analysis is finished.

1. Knowing the treatment labels can influence discretionary exclusions or the choice of a favorable outcome summary. 2. Such analysis decisions can systematically favor the expected treatment result even though assignment was randomized. 3. Define exclusion rules and the primary outcome before examining treatment differences. 4. When possible, give the analyst neutral group codes and reveal the treatment identities only after the planned analysis is complete.

Unmasked analysis can introduce treatment-favoring decisions after the data are seen. Predefine exclusions and outcomes, and analyze neutrally coded groups before revealing which code is the new treatment.

- Ask whether participants can keep their assigned procedures separate. - Identify the natural group within which information is shared. - Choose an assignment unit that keeps each treatment condition intact.

1. Teammates assigned different protocols can exchange instructions and influence one another's behavior. 2. Individual treatment conditions may therefore mix, reducing the intended contrast. 3. Randomly assigning whole teams to protocols can reduce contamination because all members of a team use the same protocol. 4. The team then becomes the experimental unit, and replication is based on the number of independently assigned teams.

Individual assignments within teams may contaminate each other through shared procedures. Randomly assign entire teams to protocols; teams, not employees, are then the experimental units.

- List what changes from the beginning to the end of the measurement session. - Compare that sequence with the treatment order. - Break the link by mixing treatment groups throughout the measurement process.

1. Treatment is linked to measurement time because every A response is measured earlier than every B response. 2. Instrument drift can create a treatment difference even if the treatments have the same true effect. 3. Random assignment of treatment to specimens does not remove bias introduced later by a fixed measurement order. 4. Randomize or interleave the measurement order across treatment groups and recalibrate the instrument on a planned schedule.

Treatment is confounded with measurement time and instrument drift. Randomize or alternate the measurement order across treatments and use scheduled calibration checks.

- Ask whether the effect of the first treatment can wear off before the second. - Consider whether the same response can be measured independently twice. - Choose a design in which each student receives only one lasting instructional treatment.

1. Learning from the first semester is lasting and changes the student's knowledge before the second treatment. 2. The response under the second method cannot be separated from knowledge gained under the first method. 3. Randomizing order does not remove this permanent carryover. 4. A better design randomly assigns comparable students to one method for the same course period, possibly using baseline achievement blocks.

The first course permanently changes the student's algebra knowledge, so the second response is contaminated by prior learning. Use parallel treatment groups, with random assignment and optional blocking by baseline achievement.

- Ask what random assignment accomplishes before outcomes are measured. - Distinguish a valid analysis choice from a required analysis choice. - Consider when the response definition should be selected.

1. Random assignment makes a comparison of final outcomes a valid way to estimate the treatment effect. 2. Comparing change scores is also possible when baseline and final scores are available. 3. Similar baseline means do not make final-score comparison invalid, and random assignment does not require using only one of these response definitions. 4. The primary outcome and analysis should be chosen in advance rather than selected after seeing which comparison is more favorable.

The claim is false. Random assignment permits a comparison of final scores, and a comparison of change scores is another possible preplanned analysis. The study should specify its primary response and analysis before examining treatment results.

- Track when chance determined a participant’s group and when personal choice entered. - Ask whether the final groups still have the composition created at the start. - Consider which grouping keeps the original randomization intact.

1. Switching depends on participants’ experiences and preferences, so the completed-program groups are partly self-selected. 2. Comparing those groups no longer preserves the balance created by random assignment. 3. A better analysis compares participants according to their original random assignments; the design may also restrict switching or record it as an additional outcome.

The completed-program groups are not fully randomized because switching is self-selected. The primary comparison should use the original assigned groups so that the benefit of random assignment is preserved.