ETC5521 Worksheet Week 8

Going beyond two variables, exploring high dimensions

Exercise 1: Risk taking on vacation

The book “Market Segmentation Analysis - Understanding It, Doing It, and Making It Useful” by Sara Dolnicar, Bettina Grün and Friedrich Leisch has a selection of useful data sets. These can be used by installing the package:

install.packages("https://statistik.boku.ac.at/nachlass_leisch/MSA/packages/MSA_0.3-1.tar.gz", repos = NULL, type = "source")

If you can’t install the package, you can download the data from the book website.

Load the risk data. This data has 563 observations and 6 variables.

The data was collected by academic researchers using a permission based online panel.

The sample was taken from adult Australian residents who have undertaken at least one holiday in the last year which involved staying away from home for at least four nights.

The respondents were asked: “Which risks have you taken in the past?” and answered on a 5-point scale with options:

• Never (1)
• Rarely (2)
• Quite often (3)
• Often (4)
• Very often (5)

The six types of risk, which form the variables were:

• Recreational risks (e.g., rock-climbing, scuba diving)
• Health risks (e.g., smoking, poor diet, high alcohol consumption)
• Career risks (e.g., quitting a job without another to go to)
• Financial risks (e.g., gambling, risky investments)
• Safety risks (e.g., speeding)
• Social risks (e.g., standing for election, publicly challenging a rule or decision)

1. Use AI to find an explanation of market segmentation. Write a few sentences to explain it.

2. What type of plots should you make for this data?

3. What this doesn’t consider is whether some customers respond similarly to other customers. How would you check this?

4. A common approach to do market segmentation is to conduct a cluster analysis (outside the scope of this class), which will divide the observations into groups. The question becomes “how many groups would be suitable to summarise the customer behaviour?” We are going to use the guided tour to help make the decision.

This is code that will do the clustering:

risk_d <- apply(risk, 2, function(x) (x - mean(x)) / sd(x))

# Clustering
nc <- 2 # Set the number of clusters
set.seed(1145)
r_km <- kmeans(risk_d, centers = nc, iter.max = 500, nstart = 5)

r_km_d <- risk_d |>
  as_tibble() |>
  mutate(cl = factor(r_km$cluster)) |>
  bind_cols(model.matrix(~ as.factor(r_km$cluster) - 1))
colnames(r_km_d)[(ncol(r_km_d) - nc + 1):ncol(r_km_d)] <- paste0(
  "cluster",
  1:nc
)
r_km_d <- r_km_d |>
  mutate_at(vars(contains("cluster")), function(x) x + 1)

Change the number of clusters nc starting with 2 up to maybe 5. Examine how it is breaking the data, and what the groups would mean using the 6 types of risk.

animate_xy(r_km_d[, 1:6], guided_tour(lda_pp(r_km_d$cl)), col = r_km_d$cl)

Exercise 2: Parkinsons

This dataset is composed of a range of biomedical voice measurements from 31 people, 23 with Parkinson’s disease (PD). Each column in the table is a particular voice measure, and each row corresponds one of 195 voice recording from these individuals (“name” column). The main aim of the data is to discriminate healthy people from those with PD, according to “status” column which is set to 0 for healthy and 1 for PD.

The data is available at The UCI Machine Learning Repository in ASCII CSV format. The rows of the CSV file contain an instance corresponding to one voice recording. There are around six recordings per patient, the name of the patient is identified in the first column. There are 24 variables in the file, including the persons name in column 1.

The data are originally analysed in: Max A. Little, Patrick E. McSharry, Eric J. Hunter, Lorraine O. Ramig (2008), ‘Suitability of dysphonia measurements for telemonitoring of Parkinson’s disease’, IEEE Transactions on Biomedical Engineering (to appear).

library(cassowaryr)
# Load the data
data(pk)

1. How many pairwise plots would you need to look at, to look at all of them?

2. Compute several of the scagnostics (monotonic, outlying, clumpy2) for the first five variables of variables, except for name. (Note: We are using just five for computing speed, but the scagnostics could be calculated on all variables.)

# Compute the scagnostics on the relevant variables
s <- calc_scags_wide(pk[,2:5],
                scags=c("outlying","monotonic",
                        "clumpy2"))
s

3. Sort the scagnostics, separately by the values on (i) monotonic (ii) outlying (iii) clumpy2, and plot the pair of variables with the highest values on each.

4. Make an interactive scatterplot matrix. Browse over it to choose other interesting pairs of variables and make the plots.

5. The scagnostics help us to find interesting associations between pairs of variables. However, the problem here is to detect differences between Parkinsons’ patients and normal patients. How would you go about that? Think about some ideas long the line of scagnostics but look for differences between the two groups.

6. Now try to do this using the scagnostics.