I wonder if you’d be ok to help me to understanding this Gelman’s graph. I struggle to understand what is the plotted distribution and the exact meaning of the red area. Of course I read the related article, but it doesn’t help me much.

We frame our calculations not in terms of Type 1 and Type 2 errors but rather Type S (sign) and Type M (magnitude) errors, which relate to the probability that claims with confidence have the wrong sign or are far in magnitude from underlying effect sizes (p. 2)

*must*have exaggerated effect sizes and are likely to have the wrong sign. The graph in question:

### Let’s build our own Type-S and Type-M graph

**R**

**code to make all figures is provided at the end of this post (except the gif).**

*any statistically significant result must have a sample mean that is much larger in magnitude than the true population mean,*and is quite likely to have the wrong sign.

setting the true effect size to 2% and the standard error of measurement to 8.1%, the power comes out to 0.06, the Type S error probability is 24%, and the expected exaggeration factor is 9.7. Thus, it is quite likely that a study designed in this way would lead to an estimate that is in the wrong direction, and if “significant,” it is likely to be a huge overestimate of the pattern in the population. (p. 6)

*use*repeated-sampling statistical inference. This is simply an explanation of the concept. Be a Bayesian!)

### R code

#The first plot with the null value and the proposed true value | |

x <- seq(-35,35,.001) #set up for plotting the curve | |

y <- dnorm(x,0,8.1) #y values for plotting curve | |

plot(x=x,y=y, main="Type-S and Type-M error example", xlab="Estimated effect size", | |

ylab="Relative frequency", type="l", cex.lab=1.5, axes=F, col="white") | |

axis(1,at=seq(-30,30,10),pos=0) #make the axis nice | |

axis(2,at=seq(0,.05,.01),pos=-35,las=1) #make the axis nice | |

lines(c(0,0),c(0,.05),col="red",lwd=3) ##Add line at null value | |

lines(c(2,2),c(0,.05),col="blue",lwd=3) ##Add line at population mean | |

points(17, .001, pch=23, bg="grey",col="black",cex=1.5) ##Add sample mean | |

####################################################################################################### | |

##The second and third plots with the null sampling distribution and significance areas under the curve | |

x <- seq(-35,35,.001) #set up for plotting the curve | |

y <- dnorm(x,0,8.1) #y values for plotting curve | |

plot(x,y, main="Type-S and Type-M error example", xlab="Estimated effect size", | |

ylab= "Relative frequency", type="l",cex.lab=1.5, las=1, lwd=3, axes = F) | |

axis(1,at=seq(-30,30,10),pos=0) #make the x axis nice | |

axis(2,at=seq(0,.05,.01),pos=-35,las=1) #make the y axis nice | |

lines(c(0,0),c(0,dnorm(0,0,8.1)),col="red",lwd=3) ##adds null line | |

lines(c(2,2),c(0,dnorm(2,0,8.1)),col="blue",lwd=3) ##adds true pop mean line | |

points(17, .001, pch=23, bg="grey",col="black",cex=1.5) ##adds sample mean | |

##Adds shaded area | |

cord.x <- c(-35, seq(-35,-15.9,.01),-15.9) ##set up for shading | |

cord.y <- c(0,dnorm(seq(-35,-15.9,.01),0,8.1),0) ##set up for shading | |

polygon(cord.x,cord.y,col='red') ##shade left tail | |

cord.xx <- c(35, seq(35,15.9,-.01),15.9) | |

cord.yy <- c(0,dnorm(seq(35,15.9,-.01),0,8.1),0) | |

polygon(cord.xx,cord.yy,col='red') ##shade right tail | |

points(17, .001, pch=23, bg="grey",col="black",cex=1.5) ##replots the sample mean over the shading | |

####################################################################################################### | |

##The fourth plot with the alternative sampling distribution and significance areas under the curve | |

x <- seq(-35,35,.001) #set up for plotting the curve | |

y <- dnorm(x,2,8.1) #y values for plotting curve | |

plot(x,y, main="Type-S and Type-M error example", xlab="Estimated effect size", | |

ylab= "Relative frequency", type="l", cex.lab=1.5, las=1, lwd=3, axes = F) | |

axis(1,at=seq(-30,30,10),pos=0) #make the x axis nice | |

axis(2,at=seq(0,.05,.01),pos=-35, las=1) #make the y axis nice | |

lines(c(0,0),c(0,dnorm(0,2,8.1)),col="red",lwd=3) ##add vertical line at null value | |

lines(c(2,2),c(0,dnorm(2,2,8.1)),col="blue",lwd=3) ##add vertical line at population mean | |

cord.x <- c(-35, seq(-35,-15.9,.01),-15.9) ##set up for shading | |

cord.y <- c(0,dnorm(seq(-35,-15.9,.01),2,8.1),0) ##set up for shading | |

polygon(cord.x,cord.y,col='red') ##shade left tail | |

cord.xx <- c(35, seq(35,15.9,-.01),15.9) | |

cord.yy <- c(0,dnorm(seq(35,15.9,-.01),2,8.1),0) | |

polygon(cord.xx,cord.yy,col='red') ##shade right tail | |

points(17, .001, pch=23, bg="grey",col="black", cex=1.5) ##replots sample mean over shading |