ETC5521 Tutorial 1

Introduction to exploratory data analysis

Author

Prof. Di Cook

🎯 Objectives

This is the first tutorial meeting of the semester. The goal is to get to know other people in the class with you, and your tutors, and check you’ve got the right skills to get started, and to begin thinking about exploratory data analysis.

🔧 Preparation

  • Have git installed on your laptop so that you can access the GitHub classroom.
  • Have the latest versions of RStudio and R installed on your laptop.
  • Install this list of R packages:
  • Create an RStudio project for this unit, called ETC5521. All your work in the tutorials should be conducted in this project. Ideally, your project is organised into folders, one for data, one for tutorial_XX, … Each week when you begin your tutorial, open the project.
  • Alternatively, you can use positron, but make sure to keep everything organised similarly to the above structure.

📥 Exercises

1. How good are your detective skills?

Being good at noticing something unexpected or unusual is an important skills for exploratory data analysis. This exercise is designed to practice your detective skills.

Play the game alzheimer_test from the fun package by running this code:

library(fun)
x = alzheimer_test()

You will be given 6 tasks to complete. Each one is to find a specific letter hidden among a \(10\times 30\) grid of letters. Each time you will also be asked to enter the row, and then the column of your answer. You can take as much time to work these out as you need. When you are finished, answer these questions:

  1. Which task did you THINK was the most difficult?
  2. Your data was saved into the object x. Which task does the DATA say was most difficult based, based on the time taken to answer, which is variable tm1.1.j. in your results data?
  3. Save the dataset to alzheimers.rda file, and share with your tutor.
load(here::here("data/alzheimers.rda"))
library(dplyr)
x |>
  select(char1.1.j., char2.1.j., tm1.1.j.) |>
  arrange(desc(tm1.1.j.))
           char1.1.j. char2.1.j.  tm1.1.j.
ans.user.2          M          N 30.839718
ans.user.3          I          T 19.695932
ans.user.5          D          O 17.189302
ans.user.1          O          C 16.534676
ans.user.4          F          E  4.424869
ans.user            9          6  3.812386

2. Are you like everyone else?

With the combined data collated by the tutor, answer these questions:

  1. What was the task that was the most difficult based on the overall average time?
  2. Which task had the most variation among the class?
  3. Is there a relationship between the location of the hidden letter and the time taken to find it? (How should you calculate this?)
  4. How fast are you relative to your class mates? (How should you calculate this?)

The code to be able to answer the questions above is:

load(here::here("data/alzheimers-class.rda"))
xclass |>
  group_by(char2.1.j.) |>
  summarise(m = mean(tm1.1.j.)) |>
  arrange(desc(m))
# A tibble: 6 × 2
  char2.1.j.     m
  <chr>      <dbl>
1 N          20.8 
2 O          15.1 
3 T           7.75
4 C           5.99
5 6           5.18
6 E           4.11
xclass |>
  group_by(char2.1.j.) |>
  summarise(s = sd(tm1.1.j.)) |>
  arrange(desc(s))
# A tibble: 6 × 2
  char2.1.j.     s
  <chr>      <dbl>
1 N          17.0 
2 O          13.0 
3 T           7.11
4 C           6.01
5 6           3.09
6 E           2.35

The variables X1.1, X2.1 have the true location of the letters. Row might be easier to compute than column, because there are only 6 rows. With 30 columns, there are too many location options to check. Grouping into bunches of 5 might help.

library(ggplot2)
xclass |> 
  group_by(X1.1) |>
  summarise(m = mean(tm1.1.j.)) |>
  ggplot(aes(X1.1, m)) + 
    geom_point() +
    xlab("Row") +
    ylab("Time")

Should you adjust by the letter? Probably, because some letters might just be harder to find than others, regardless of where they are hidden. If certain letters happen to have been placed more often in particular rows, then those rows would look artificially slow or fast, when really it’s the letter that’s driving the difference. Adjusting for letter removes this confounding.

For part (d), the challenge is that each of the 6 tasks has a different average time and a different amount of spread (some tasks are just harder, or more variable, than others). This means a raw time of, say, 10 seconds could be fast for one task and slow for another, so raw times aren’t directly comparable across tasks. Standardising (subtracting the task mean and dividing by the task standard deviation, i.e. a z-score) puts every task on the same scale: it tells you how many standard deviations above or below the class average you were on that task, regardless of how hard the task was overall. Once every task is on this common scale, you can fairly compare your performance across tasks, and even average your standardised scores together to get one overall measure of relative speed. In the sample data there are only two participants, which is not enough to compute standardised scores meaningfully.

tasks <- xclass |>
  group_by(char2.1.j.) |>
  summarise(m = mean(tm1.1.j.),
            s = sd(tm1.1.j.))
myresults <- xclass |>
  filter(id == 1) # Use your id here!
myresults |>
  left_join(tasks, by = "char2.1.j.") |>
  mutate(std_score = (tm1.1.j. - m)/s) |>
  select(char2.1.j., std_score)
  char2.1.j.  std_score
1          6 -0.4426827
2          C  1.7538244
3          N  0.5898853
4          T  1.6799758
5          E  0.1338864
6          O  0.1582762

3. Share your GitHub names with your tutor

Please provide your GitHub username to your tutor, using the Google sheet provided. See this sheet.

👌 Finishing up

Make sure you say thanks and good-bye to your tutor. This is a time to also report what you enjoyed and what you found difficult.