ETC5521 Tutorial 5

Working with a single variable, making transformations, detecting outliers, using robust statistics

🎯 Objectives

These are exercises in making plots of one variable and what can be learned about the distributions, data patterns and apply randomisation methods to check what we see.

🔧 Preparation

The reading for this week is Wilke (2019) Ch 6 Visualizing Amounts; Ch 7 Visualizing distributions. - Make sure you have this list of R packages installed:

install.packages(c("MASS", "ggbeeswarm", "patchwork", "nullabor", "lvplot"))

• Open your RStudio Project for this unit, (the one you created in week 1, ETC5521). Create a .qmd document for this weeks activities.

📥 Exercises

1. Understanding the velocity of galaxies

Load the galaxies data in the MASS package and answer the following questions based on this dataset.

data(galaxies, package = "MASS")

You can access documentation of the data (if available) using the help function specifying the package name in the argument.

help(galaxies, package = "MASS")

1. What does the data contain? And what is the data source?

SolutionSolution

data(galaxies, package = "MASS")
glimpse(galaxies)

 num [1:82] 9172 9350 9483 9558 9775 ...

The data contains velocities in km/sec of 82 galaxies from 6 well-separated conic sections of an unfilled survey of the Corona Borealis region. The original data is from Postman et al. (1986) and this data is from Roeder with 83rd observation removed from the original data as well as typo for the 78th observation.

• Postman, M., Huchra, J. P. and Geller, M. J. (1986) Probes of large-scale structures in the Corona Borealis region. Astronomical Journal 92, 1238–1247
• Roeder, K. (1990) Density estimation with confidence sets exemplified by superclusters and voids in galaxies. Journal of the American Statistical Association 85, 617–624.

2. Based on the description in the R Help for the data, what would be an appropriate null distribution of this data?

SolutionSolution

The description in the R help for the data says Multimodality in such surveys is evidence for voids and superclusters in the far universe.

Deciding on an appropriate null hypothesis is always tricky. If we wanted to test the statement that the data is multimodal, we could compare against a unimodal distribution, either a normal or an exponential depending on what shape we might expect.

However, the published work has already made a claim that the data is multimodal, so it would be interesting to determine if we can generate samples from a multimodal distribution that are indistinguishable from the data.

\(H_0:\) The distribution is multimodal. \(H_a:\) The distribution is something other than multimodal.

3. How many observations are there?

SolutionSolution

There are 82 observations.

4. If the data is multimodal, which of the following displays do you think would be the best? Which would not be at all useful?

• histogram
• boxplot
• density plot
• violin plot
• jittered dot plot
• letter value plot

SolutionSolution

If you said a density plot, jittered dot plot, or a histogram, you’re on the right track, because each can give a fine resolution for showing modes. (The violin plot is not any different from a density plot, when only looking at one variable.)

5. Make these plots for the data. Experiment with different binwidths for the histogram and different bandwiths for the density plot. Were you right in your thinking about which would be the best?

SolutionSolution

g <- ggplot(tibble(galaxies), aes(galaxies)) +
  theme(
    axis.title = element_blank(),
    axis.text = element_blank(),
    axis.ticks = element_blank()
  )
g1 <- g + geom_histogram(binwidth = 1000, color = "white") 
g2 <- g + geom_boxplot() 
g3 <- g + geom_density() 
g4 <- g + geom_violin(aes(x=galaxies, y=1), draw_quantiles = c(0.25, 0.5, 0.75))
g5 <- g + geom_quasirandom(aes(x=1, y=galaxies)) + coord_flip() 
g6 <- g + geom_lv(aes(x=1, y=galaxies)) + coord_flip() 

g1 + g2 + g3 + g4 + g5 + g6 + plot_layout(ncol = 2)

6. Fit your best mixture model to the data, and simulate 19 nulls to make a lineup. Did you do a good job in matching the distribution, ie does the data plot stand out or not? (Extra bonus: What is the model that you have created? Can you make a plot to show how it looks relative to the observed data?)

This code might be helpful to get you started. This code generates a jittered dotplot, but you can use your preferred type from part e.

# Fit a mixture model
library(mixtools)
galaxies_fit <- normalmixEM(galaxies, k=3)

set.seed(1138)
galaxies_sim1 <- rnormmix(n=length(galaxies), 
              lambda=galaxies_fit$lambda, 
              mu=galaxies_fit$mu,
              sigma=galaxies_fit$sigma)

# Plot your data
ggplot(tibble(galaxies_sim1), aes(x=galaxies_sim1)) +
  geom_quasirandom(aes(x=1, y=galaxies_sim1)) + 
  coord_flip() +
  theme(
    aspect.ratio = 0.7,
    axis.title = element_blank(),
    axis.text = element_blank(),
    axis.ticks = element_blank()
  )

# Generate null plots and make a lineup
galaxies_null <- tibble(.sample=1, galaxies=galaxies_sim1)
for (i in 2:19) {
  gsim <- rnormmix(n=length(galaxies), 
              lambda=galaxies_fit$lambda, 
              mu=galaxies_fit$mu,
              sigma=galaxies_fit$sigma)
  galaxies_null <- bind_rows(galaxies_null,
                             tibble(.sample=i, galaxies=gsim))
}
galaxies_null <- bind_rows(galaxies_null,
                             tibble(.sample=20,
                                    galaxies=galaxies))
# Randomise .sample  to hide data plot
galaxies_null$.sample <- rep(sample(1:20, 20), rep(82, 20))
ggplot(tibble(galaxies_null), aes(x=galaxies)) +
  geom_quasirandom(aes(x=1, y=galaxies)) + 
  facet_wrap(~.sample, ncol=5) +
  coord_flip() +
  theme(
    aspect.ratio = 0.7,
    axis.title = element_blank(),
    axis.text = element_blank(),
    axis.ticks = element_blank()
  )

SolutionSolution

number of iterations= 78 

# To make a rough plot of your model
plot(galaxies_fit, whichplots=2)

The lambda value provides the proportion of mixing, from three normal samples. The mu and sigma give the mean and standard deviations for each of the distributions.

2. What is the best transformation to make?

For each of the variables in the data, which-transform.csv, decide on an appropriate transformation to make the distribution more symmetric for five of the variables and remove discreteness on one variable.

SolutionSolution

Remember the power ladder, go down to fix right-skew, and up to fix left-skew. For multi-modal find an explanatory variable, or do a severe quantile transformation.

👌 Finishing up

Make sure you say thanks and good-bye to your tutor. This is a time to also report what you enjoyed and what you found difficult.