ETC5521 Tutorial 4

Computational evidence

Author

Prof. Di Cook

🎯 Objectives

Practice conducting using computational tools to assess significance of patterns.

🔧 Preparation

The reading for this week is Wickham et al. (2010) Graphical inference for Infovis.
- Make sure you have this package installed from github:

devtools::install_github("kevinwang09/learningtower")
  • Open your RStudio Project for this unit, (the one you created in week 1, ETC5521). Create a .qmd document for this weeks activities.

📥 Exercises

Exercise 1

Load the 2022 data, and filter on Australia. Make a quick check on the missng values, and the variables contained in the data. Here is the code to load the data:

stu_2022 <- load_student(year = "2022")
stu_2022_oz <- stu_2022 |>
  filter(country == "AUS")

There are 13437 observations.

library(naniar)
vis_miss(stu_2022_oz, sort_miss = TRUE)

The variables desk, dishwasher and wealth are completely missing. Otherwise, there are a few sporadic missings on all of the other demographic variables. The scores, schools, country, year variables are complete.

glimpse(stu_2022_oz)
Rows: 13,437
Columns: 22
$ year        <int> 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022…
$ country     <chr> "AUS", "AUS", "AUS", "AUS", "AUS", "AUS", "AUS", "AUS", "A…
$ school_id   <chr> "3600477", "3600016", "3600669", "3600495", "3600325", "36…
$ student_id  <chr> "3600001", "3600002", "3600004", "3600006", "3600008", "36…
$ mother_educ <fct> "ISCED 3A", "ISCED 3A", "ISCED 2", NA, "ISCED 2", "ISCED 3…
$ father_educ <fct> "ISCED 3A", "ISCED 3A", "ISCED 2", NA, "ISCED 3B, C", "ISC…
$ gender      <fct> male, female, male, male, female, female, female, female, …
$ computer    <fct> yes, yes, yes, NA, NA, yes, yes, yes, yes, yes, yes, yes, …
$ internet    <fct> yes, yes, yes, yes, yes, yes, yes, yes, yes, yes, yes, yes…
$ math        <dbl> 447, 582, 350, 281, 364, 593, 511, 516, 447, 354, 486, 424…
$ read        <dbl> 350, 646, 204, 285, 347, 544, 611, 545, 473, 429, 483, 298…
$ science     <dbl> 560, 531, 375, 385, 350, 589, 595, 552, 475, 370, 508, 423…
$ stu_wgt     <dbl> 21.5, 26.4, 34.4, 30.6, 8.9, 22.7, 17.3, 46.6, 3.9, 12.3, …
$ desk        <lgl> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA…
$ room        <fct> yes, yes, yes, yes, yes, yes, yes, yes, yes, yes, yes, no,…
$ dishwasher  <lgl> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA…
$ television  <fct> 2, 1, 2, NA, 2, 2, 2, 1, 2, 2, 3+, 1, 1, 1, NA, 2, 1, 2, 1…
$ computer_n  <fct> 3+, 3+, 3+, NA, NA, 3+, 3+, 3+, 3+, 2, 3+, 2, 3+, 3+, NA, …
$ car         <fct> 3+, 2, 1, 3+, 3+, 3+, 3+, 2, 2, 2, 3+, 3+, 1, 2, NA, 3+, 2…
$ book        <fct> 201-500, 101-200, 0-10, 0-10, 101-200, 11-25, 101-200, 101…
$ wealth      <lgl> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA…
$ escs        <dbl> 0.97, 1.39, 0.78, NA, -1.02, 0.34, -0.11, 0.84, 0.82, -0.2…

Exercise 2

Pick one of the categorical variables, and one of the score variables. Write down the ggplot2 code to make a plot of your chosen variables. Then answer these questions:

  1. What would be the obvious null hypothesis associated with your plot? That is, write out in English what it would mean to say that there is nothing interesting to see in the plot.
  2. Then what would be the alternative hypothesis?
  3. From your null hypothesis, what would be a suitable null data generating mechanism?
  4. Adjust your code to make a lineup plot, and use yourself to test whether the hull hypothesis can be rejected or not.
ggplot(stu_2022_oz, aes(x=television, 
                        y=read)) +
  geom_quasirandom(alpha=0.5) +
  stat_summary(colour = "red")
  1. The null hypothesis would be that there is no relationship between number of TVs in the household and the reading scores.
  2. The alternative hypothesis is that there is a relationship.
  3. A suitable null generating mechanism is the permute one of the variables. It could be either.
set.seed(906)
ggplot(lineup(null_permute("read"), stu_2022_oz), 
    aes(x=television, 
        y=read)) +
  geom_quasirandom(alpha=0.5) +
  stat_summary(colour = "red") +
  facet_wrap(~.sample)

For this example, we can see the difference between the means across the number of TVs in one plot but not the others, and when you run the decrypt line it matches the data plot. So here we would reject the null hypothesis and conclude that there is a relationship between reading scores and number of TVs.

Exercise 3

This question is to assess if your choice of plot type might affect the ability to detect the difference. Whatever your conclusion from the activity in the previous question, think about an alternative plot design.

  1. Make the lineup with the different plot design. (You might use a new position - by not setting a seed and re-generating the lineup - every time you show someone, so if they hear another class mate say a number it won’t influence the next person.)
  2. Show half of your class mates, the new design, and half the old design.
  3. Record how many of them chose the data plot.

Here I am generating the same plot, without the mean summary overlaid. I think it is harder to see the difference between plots.

ggplot(lineup(null_permute("read"), stu_2022_oz), 
    aes(x=television, 
        y=read)) +
  geom_quasirandom(alpha=0.5) +
  stat_summary(colour = "red") +
  facet_wrap(~.sample)

ggplot(lineup(null_permute("read"), stu_2022_oz), 
    aes(x=television, 
        y=read)) +
  geom_quasirandom(alpha=0.5) +
  #stat_summary(colour = "red") +
  facet_wrap(~.sample)

This comparison actually raises an important question: are these two plots testing the same null hypothesis?

Technically, the answer is NO, emphatically! when you overlay the mean, with a coloured point, the plot is now focusing on the differences between means. A more appropriate comparison is to change the way the mean is represented:

ggplot(lineup(null_permute("read"), stu_2022_oz), 
    aes(x=television, 
        y=read)) +
  geom_quasirandom(alpha=0.5) +
  stat_summary(geom = "crossbar", width = 0.5,
    colour = "red") +
  facet_wrap(~.sample)

Exercise 4

The PISA data is actually an example of survey data, which means each observation is associated with a weight (stu_wgt). We need to work out how this should be incorporated in the the analysis.

  1. Find out what you can about this variable and how it should be used in an analysis, using AI if needed, or reading the documentation.
  2. How would you calculate a mean, for survey data like this that has a weight variable?
  3. What plotting methods can accommodate weights?

Here is a small summary of the weights by gender and school. We can see that they are varied.

stu_2022_oz |> 
  group_by(gender, school_id) |>
  summarise(m = mean(stu_wgt, na.rm=TRUE),
            s = sd(stu_wgt, na.rm=TRUE)) |>
  ggplot(aes(x=gender, y=m)) +
    geom_quasirandom(alpha = 0.5)

  1. The weight variable takes various survey design features into account, so that any estimates might better reflect the population. The population is all school children of this age group being tested. The weight accommodates school sampling, and sampling of students within the school, along with demographics like gender, and year level relative to age.

  2. You would use a weighted_mean() function instead of mean().

  3. There are a few geoms for single variable plots that can use the weight appropriately, e.g. geom_density(). Make the plot with and without adjusting with the weight. Explain any difference.

library(ggridges)
library(patchwork)
p1 <- ggplot(stu_2022_oz, 
    aes(x=read, 
        y=television)) +
  geom_density_ridges(
    fill = "red", 
    colour = "red",
    alpha = 0.5) 
    
p2 <- ggplot(stu_2022_oz, 
    aes(x=read, 
        y=television, 
        weight=stu_wgt)) +
  geom_density_ridges(
    fill = "red", 
    colour = "red",
    alpha = 0.5) 

p1 + p2 + plot_layout(ncol=2)

There is a small amount of difference in the density plots made here. It might be argued that it is not large enough to change any decisions.

👌 Finishing up

Make sure you say thanks and good-bye to your tutor. This is a time to also report what you enjoyed and what you found difficult.