For cross-text questions involving apparently conflicting results, identify exactly what population and statistic each text describes. Then check whether the second text introduces a grouping or selection effect that can reconcile the results without accepting the first text's proposed explanation.
Hints
- Hint 1
Compare the measured groups
Consider which applicants are included in a statistic based only on applications that reach submission.
- Hint 2
Distinguish grouped and overall results
Ask whether an outcome that holds within every project-complexity group must also appear in the same way when all submitted applications are combined.
Step-by-step
- Step 1
Identify the proposed explanation
The researchers attribute the conflicting results to the setting: they suggest that the supervised usability sessions may not represent independent use of the application.
- Step 2
Separate within-group results from the overall statistic
In Text 2, the redesign reduces submission time within every project-complexity group. It also prevents more applicants with highly complex, generally time-consuming projects from abandoning their applications.
- Step 3
Account for the changed group of submitted applications
Abandoned applications are not included in a median calculated from submitted applications. If the redesign enables more complex applications to reach submission, those comparatively long completion times enter the data and can raise the overall median even though applicants complete the redesigned version faster within each complexity group. Text 2 therefore supplies an alternative to the supervision-based explanation.