Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Jan-Dec:23:2331216519832483.
doi: 10.1177/2331216519832483.

Analyzing the Time Course of Pupillometric Data

Jacolien van Rij  1 , Petra Hendriks  1 , Hedderik van Rijn  1 , R Harald Baayen  2 , Simon N Wood  3
Affiliations

Analyzing the Time Course of Pupillometric Data

Jacolien van Rij et al. Trends Hear. 2019 Jan-Dec.

Abstract

This article provides a tutorial for analyzing pupillometric data. Pupil dilation has become increasingly popular in psychological and psycholinguistic research as a measure to trace language processing. However, there is no general consensus about procedures to analyze the data, with most studies analyzing extracted features from the pupil dilation data instead of analyzing the pupil dilation trajectories directly. Recent studies have started to apply nonlinear regression and other methods to analyze the pupil dilation trajectories directly, utilizing all available information in the continuously measured signal. This article applies a nonlinear regression analysis, generalized additive mixed modeling, and illustrates how to analyze the full-time course of the pupil dilation signal. The regression analysis is particularly suited for analyzing pupil dilation in the fields of psychological and psycholinguistic research because generalized additive mixed models can include complex nonlinear interactions for investigating the effects of properties of stimuli (e.g., formant frequency) or participants (e.g., working memory score) on the pupil dilation signal. To account for the variation due to participants and items, nonlinear random effects can be included. However, one of the challenges for analyzing time series data is dealing with the autocorrelation in the residuals, which is rather extreme for the pupillary signal. On the basis of simulations, we explain potential causes of this extreme autocorrelation, and on the basis of the experimental data, we show how to reduce their adverse effects, allowing a much more coherent interpretation of pupillary data than possible with feature-based techniques.

Keywords: autocorrelation; generalized additive mixed model; preprocessing; pupillometry; statistical analysis.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Properties of pupil dilation. Left: The effect of cognitive processing on pupil dilation, as described by the pupil dilation function from (Hoeks & Levelt, 1993, p. 21). The pupillary response is scaled to 0.5 mm for comparison (cf. Beatty & Lucero-Wagoner, 2000). The red vertical line T1 represents an event that triggers dilation (black solid line). The dashed line shows the adjusted dilation when a second event T2 shortly follows the first event. Right: Example of two actual recorded pupil dilation time series, recorded from two different participants (solid vs. dashed lines) in two different trials (red vs. black lines). The data are realigned on the onset of the pronoun. The horizontal bars indicate the duration of the auditory stimuli (two spoken sentences) of the two trials.
Figure 2.
Figure 2.
Baseline correction and normalization. Top: Three pupil dilation trials (simulated data) with Trials 1 and 3 differing in baseline, but showing the exact same pupillary response. Trials 2 and 3 share the same baseline, but Trial 2 shows a higher peak amplitude. Left: The same three trials after the baseline are subtracted. The baselined data for Trials 1 and 3 overlap. Right: The proportion pupil dilation change with respect to the baseline for the same three trials. As the baseline of Trial 1 is much higher than of Trial 3, the pupil dilation change is much lower for Trial 1 than for Trial 3, although the measured pupillary response was exactly the same.
Figure 3.
Figure 3.
Example visual materials (see sentences in Table 1): congruent with test sentence (left) and incongruent with test sentence (right).
Figure 4.
Figure 4.
Left: Example of two pupil dilation time series, recorded from two different participants (solid vs. dashed lines) in two different trials (red vs. black lines). The data are aligned on the onset of the pronoun. The horizontal bars indicate the duration of the auditory stimuli of the two trials, which consisted of two sentences. The baseline for the averages in this graph was calculated from a 250-ms time window before sound onset. Right: The grand averages for the four conditions. In contrast with the plot earlier, the baseline window of this analysis data was at the pronoun onset, indicated with the vertical line.
Figure 5.
Figure 5.
Partial effects (fixed effects only) of the initial GAMM. The top four panels show the nonlinear regression lines for each of the four conditions with pointwise 95% confidence intervals, with the value of the parametric estimates for that condition on the right (red numbers). The bottom panel shows the interaction between Xgaze and Ygaze and implements the effect of gaze position on the measured pupil size. Note that (0,0) represents the top-left corner of the screen.
Figure 6.
Figure 6.
Schematic illustration of the three types of random effects. The y axis represents the measurement scale. The black thick line outlines the fixed effect estimate, whereas the dashed red lines illustrate how the random effects modulate the fixed effects. The bottom right panel separates the fixed effects from the random effects.
Figure 7.
Figure 7.
Random factor smooths for participants and items estimated by model, model1.
Figure 8.
Figure 8.
Estimates of the initial GAMM model1. Left: Summed effects for all conditions, with the random effects set to zero. Bottom: Difference curves, derived from model1. The gray solid line represents the estimated difference (and pointwise 95% confidence intervals) between the incongruent and congruent items when the actor is introduced first (“A1”), and the dashed red line represents the estimated difference (and pointwise 95% confidence intervals) between the incongruent and congruent items when the actor is introduced second (“A2”).
Figure 9.
Figure 9.
Residuals of the initial GAMM, model1. ACF = autocorrelation function; QQ = quantile-quantile.
Figure 10.
Figure 10.
Autocorrelation in simulation data (n = 250). For each simulation (Simulation 1 in the top row, Simulation 2 in the center row, and Simulation 3 in the bottom row), the same three plots are provided. Left: 10 randomly selected modified sine waves (of the 250 in total), and the model fit for the sine waves (red thick solid line); Center: residuals of the model for the same 10 sine waves; Right: autocorrelation of the model’s residuals for each lag (x axis). ACF = autocorrelation function.
Figure 11.
Figure 11.
Improvement in model fit by adding random smooths for unique time series. Left: Data (black thick lines) and the model fit of the initial GAMM (thick red lines) for three random three events in the experiment. Center: Data (black thick lines) and the model fit of the improved GAMM (thick red lines) for the same three time series. Right: ACF for the improved GAMM, model4. (The ACF of model1 is presented in Figure 9). ACF = autocorrelation function.
Figure 12.
Figure 12.
Determining the optimal value of ρ. Left: Decrease in fREML scores with increasing values for ρ for different models discussed in this article. Right: Correlation between model fit and data as indication for the precision of model fit plotted against the values of ρ. In both panels, the dots show the selected start values for ρ, based on the ACF Lag 1 value. fREML = fast restricted maximum likelihood.
Figure 13.
Figure 13.
Distribution of the data. QQ = quantile-quantile.
Figure 14.
Figure 14.
Evaluation of the scaled t Model 6. Top row: Estimated effects (left panel) and estimated differences with pointwise 95% confidence intervals (center and right panels). Bottom row: Residuals of model6. ACF = autocorrelation function; QQ = quantile-quantile.
See this image and copyright information in PMC

References

    1. Baayen R. H. (2010) The directed compound graph of English. An exploration of lexical connectivity and its processing consequences. In: Olson E. (ed) New impulses in word-formation (Linguistische Berichte Sonderheft 17; pp. 383–402), Hamburg, Germany: Buske.
    1. Baayen R. H., Davidson D. J., Bates D. M. (2008) Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language 59: 390–412. doi: 10.1016/j.jml.2007.12.005.
    1. Baayen R. H., van Rij J., de Cat C., Wood S. N. (2018) Autocorrelated errors in experimental data in the language sciences: Some solutions offered by generalized additive mixed models. In: Speelman D., Heylen K., Geeraerts D. (eds) Mixed effects regression models in linguistics, Berlin, Germany: Springer, pp. 49–69. . doi: 10.1007/978-3-319-69830-4_4.
    1. Baayen R. H., Vasishth S., Kliegl R., Bates D. D. M. (2017) The cave of shadows. Addressing the human factor with generalized additive mixed models. Journal of Memory and Language 94: 206–234. doi: 10.1016/j.jml.2016.11.006.
    1. Beatty J. (1982) Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological Bulletin 91(2): 276–292. - PubMed

Publication types

LinkOut - more resources