Summer School 2026
In this project, students use the performance data collected from the summer school tasks and ask a deeper question: how reliable are the measured behavioral patterns, and how should we interpret correlations when measurements are noisy?
This activity turns the tutorial into a mini research project using the class's own behavioral data.
Students complete the browser experiments and save the downloaded CSV files.
The class-level CSV files are combined across participants for each task.
Repeated measurements are split into halves to ask whether the same images or conditions produce consistent patterns.
A separate simulation notebook shows how measurement noise lowers observed correlations and how reliability can help interpret them.
Students discuss which behavioral measurements are stable, which are noisy, and what that means for comparing humans and models.
A low correlation does not always mean that two systems are unrelated. Sometimes the measurement itself is noisy.
The noise correction notebook starts with variables whose true relationship is known, then adds different amounts of measurement noise. Students can see that the observed correlation drops as the measurement gets noisier.
This prepares students to interpret real behavioral and model comparisons more carefully.
The split-half reliability notebook uses the actual task data to ask whether different random halves of the class give similar image-level or condition-level patterns.
If the split halves agree, the measurement is stable. If they disagree, we should be cautious about over-interpreting the result.
These CSV files contain the aggregate student performance used for the reliability activity.
Class responses from the object two-alternative forced-choice task.
Class responses from the drawing two-alternative forced-choice task.
Class image-rating responses used to estimate image-level consistency.
Class responses from the image memory / repetition task.
Open these in Google Colab. Save a copy to your own Drive before editing.
Use simulations to see how measurement noise weakens observed correlations, how split-half reliability is estimated, and how reliability can be used to correct correlations.
Use the class performance CSV files to estimate split-half reliability for the behavioral tasks and ask which measurements are most stable.
| Notebook | Open | Use it for |
|---|---|---|
| Noise correction tutorial | Open in Colab | Understand how noise reduces observed correlations and how reliability estimates help interpret them. |
| Reliability from task data | Open in Colab | Estimate split-half reliability from the class behavioral datasets. |
Use these prompts for a short project presentation or group discussion.
Which task produced the most reliable image-level or condition-level pattern? Which task was noisiest? What might explain the difference?
How would reliability change if each task had more participants, more repeats, or fewer stimuli?
If a human-vs-ANN correlation is low, how can we tell whether the model is poor or whether the behavioral measurement is noisy?
What would you change in the task design to make the measurements more reliable?