The data
Standardize
Effect of Standardization
At a general level
At a participant level
Distribution
Correlation
Test
Conclusion
Credits
Previous blogposts
To make sense of their data and effects, psychologists often standardize (Z-score) their variables. However, in repeated-measures designs, there are three ways of standardizing data:
Variable-wise: The most common method. A simple scaling and reducing of each variable by their mean and SD.
Participant-wise: Variables are standardized “within” each participant, i.e., for each participant, by the participant’s mean and SD.
Full: Participant-wise first and then re-standardizing variable-wise.
Unfortunately, the method used is often not explicitly stated. This is an issue as these methods can generate important discrepancies that contribute to the reproducibility crisis of psychological science.
In the following, we will see how to perform those methods and look for differences.
The data
We will take a dataset in which participants were exposed to negative pictures and had to rate their emotions (valence) and the amount of memories associated with the picture (autobiographical link). One could make the hypothesis that for young participants with no context of war or violence, the most negative pictures (mutilations) are less related to memories than less negative pictures (involving for example car crashes or sick people). In other words, we expect a positive relationship between valence (with high values corresponding to less negativity) and autobiographical link.
Let’s have a look at the data, averaged by participants:
library(tidyverse)
library(psycho)
df % # Load the dataset from the psycho package
filter(Emotion_Condition == &quot;Negative&quot;) # Discard neutral pictures
df %__% group_by(Participant_ID) %__% summarise(n_Trials = n(),
Valence_Mean = mean(Subjective_Valence, na.rm=TRUE),
Valence_SD = sd(Subjective_Valence, na.rm=TRUE),
Autobiographical_Link_Mean = mean(Autobiographical_Link, na.rm=TRUE),
Autobiographical_Link_SD = sd(Autobiographical_Link, na.rm=TRUE)) %__% mutate(Valence = paste(Valence_Mean, Valence_SD, sep=&quot; +- &quot;),
Autobiographical_Link = paste(Autobiographical_Link_Mean, Autobiographical_Link_SD, sep=&quot; +- &quot;)) %__% select(-ends_with(&quot;SD&quot;), -ends_with(&quot;Mean&quot;))
Participant_ID
n_Trials
Valence
Autobiographical_Link
10S
24
-58.07 +- 42.59
49.88 +- 29.60
11S
24
-73.22 +- 37.01
0.94 +- 3.46
12S
24
-57.53 +- 26.56
30.24 +- 27.87
13S
24
-63.22 +- 23.72
27.86 +- 35.81
14S
24
-56.60 +- 26.47
3.31 +- 11.20
15S
24
-60.59 +- 33.71
17.64 +- 17.36
16S
24
-46.12 +- 24.88
15.33 +- 16.57
17S
24
-1.54 +- 4.98
0.13 +- 0.64
18S
24
-67.23 +- 34.98
22.71 +- 20.07
19S
24
-59.61 +- 33.22
28.37 +- 12.55
As we can see from the means and SDs, there is a lot of variability between and within participants.
Standardize
We will create three dataframes standardized with each of the three techniques.
Z_VarWise % standardize()
Z_ParWise % group_by(Participant_ID) %__% standardize() Z_Full % group_by(Participant_ID) %__% standardize() %__% ungroup() %__% standardize() Effect of Standardization
Let’s see how these three standardization techniques affected the Valence variable.
At a general level
# Create convenient function
print_summary % summarise(Mean = mean(Subjective_Valence), SD = sd(Subjective_Valence)) %__% mutate_if(is.numeric, round, 2) %__% head(5) }
# Check the results
print_participants(Z_VarWise)
# A tibble: 5 x 3
Participant_ID Mean SD
1 10S -0.0500 1.15 2 11S -0.460 1.00 3 12S -0.0300 0.720
4 13S -0.190 0.640
5 14S -0.0100 0.710
print_participants(Z_ParWise)
# A tibble: 5 x 3
Participant_ID Mean SD
1 10S 0. 1.
2 11S 0. 1.
3 12S 0. 1.
4 13S 0. 1.
5 14S 0. 1.
print_participants(Z_Full)
# A tibble: 5 x 3
Participant_ID Mean SD
1 10S 0. 1.02
2 11S 0. 1.02
3 12S 0. 1.02
4 13S 0. 1.02
5 14S 0. 1.02
Distribution
data.frame(VarWise = Z_VarWise$Subjective_Valence,
ParWise = Z_ParWise$Subjective_Valence,
Full = Z_Full$Subjective_Valence) %__% gather(Method, Variable) %__% ggplot(aes(x=Variable, fill=Method)) +
geom_density(alpha=0.5) +
theme_minimal() The distributions appear to be similar…
Correlation
Let’s do a correlation between the variable-wise and participant-wise methods.
psycho::bayes_cor.test(Z_VarWise$Subjective_Valence, Z_ParWise$Subjective_Valence)
Results of the Bayesian correlation indicate moderate evidence (BF = 8.65) in favour of an absence of a negative association between Z_VarWise$Subjective_Valence and Z_ParWise$Subjective_Valence (r = -0.015, MAD = 0.047, 90% CI [-0.096, 0.057]). The correlation can be considered as small or very small with respective probabilities of 3.46% and 59.43%.
data.frame(Original = df$Subjective_Valence,
VarWise = Z_VarWise$Subjective_Valence,
ParWise = Z_ParWise$Subjective_Valence) %__% ggplot(aes(x=VarWise, y=ParWise, colour=Original)) +
geom_point() +
geom_smooth(method=&quot;lm&quot;) +
theme_minimal()
While the three standardization methods roughly present the same characteristics at a general level (mean 0 and SD 1) and a similar distribution, their values are very different and completely uncorrelated!
Test
Let’s now answer to the original question by investigating the linear relationship between valence and autobiographical link. We can do this by running a mixed model with participants entered as random effects.
# Convenient function
print_model %
summary() %__% filter(Variable == &quot;Autobiographical_Link&quot;) %__% mutate(Type = type_name,
Coef = round(Coef, 2),
p = format_p(p)) %__% select(Type, Coef, p)
}
# Run the model on all datasets
rbind(print_model(df), print_model(Z_VarWise),
print_model(Z_ParWise),
print_model(Z_Full))
Type Coef p
1 df 0.09 __ .1
2 Z_VarWise 0.07 __ .1
3 Z_ParWise 0.08 = 0.08°
4 Z_Full 0.08 = 0.08°
As we can see, in our case, using participant-wise standardization resulted in a significant (at p = .1) effect! But keep in mind that this is not always the case. In can be the contrary, or generate very similar results. No method is better or more justified, and its choice depends on the specific case, context, data and goal.
Conclusion
Standardization can be useful in some cases and should be justified
Variable and Participant-wise standardization methods produce “in appearance” similar data
Variable and Participant-wise standardization can lead to different and uncorrelated results
The choice of the method can strongly influence the results and thus, should be explicitly stated
We showed here yet another way of sneakily tweaking the data that can change the results. To prevent its use for p-hacking, we can only support the generalization of open-data, open-analysis and preregistration.
Credits
The psycho package helped you? Don’t forget to cite the various packages you used :)
You can cite psycho as follows:
Makowski, (2018). The psycho Package: An Efficient and Publishing-Oriented Workflow for Psychological Science. Journal of Open Source Software, 3(22), 470. https://doi.org/10.21105/joss.00470
Previous blogposts
Formatted Correlation with Effect Size
Extracting a Reference Grid of your Data for Machine Learning Models Visualization
Copy/paste t-tests Directly to Manuscripts
Easy APA Formatted Bayesian Correlation
Fancy Plot (with Posterior Samples) for Bayesian Regressions
How Many Factors to Retain in Factor Analysis
Beautiful and Powerful Correlation Tables
Format and Interpret Linear Mixed Models
How to do Repeated Measures ANOVAs
Standardize (Z-score) a dataframe
Compute Signal Detection Theory Indices
Installing R, R Studio and psycho