πŸ“Š Boxplots: Plotting Groups

Plotting Distributions over Categories

Qual Variables
Quant Variables
Box Plots
Violin Plots
Author

Arvind V.

Published

June 24, 2024

Modified

June 26, 2024

Abstract
Quant and Qual Variable Graphs and their Siblings

Slides and Tutorials

R (Static Viz)   Radiant Tutorial  Datasets

Setting up R Packages

options(paged.print = TRUE)
library(tidyverse)
library(mosaic)
library(ggformula)
library(palmerpenguins) # Our new favourite dataset

What graphs will we see today?

Variable #1 Variable #2 Chart Names Chart Shape
Quant Qual Box Plot

What kind of Data Variables will we choose?

No Pronoun Answer Variable/Scale Example What Operations?
1 How Many / Much / Heavy? Few? Seldom? Often? When? Quantities, with Scale and a Zero Value.Differences and Ratios /Products are meaningful. Quantitative/Ratio Length,Height,Temperature in Kelvin,Activity,Dose Amount,Reaction Rate,Flow Rate,Concentration,Pulse,Survival Rate Correlation
4 What, Who, Where, Whom, Which Name, Place, Animal, Thing Qualitative/Nominal Name Count no. of cases,Mode

Inspiration

Figure 1: Box Plot Inspiration

Alice said, β€œI say what I mean and I mean what I say!” Are the rest of us so sure? What do we mean when we use any of the phrases above? How definite are we? There is a range of β€œsureness” and β€œunsureness”…and this is where we can use box plots like Figure 1 to show that range of opinion.

How do these Chart(s) Work?

Box Plots are an extremely useful data visualization that gives us an idea of the distribution of a Quant variable, for each level of another Qual variable. The internal process of this plot is as follows:

  • make groups of the Quant variable for each level of the Qual
  • in each group, rank the Quant variable values in increasing order
  • Calculate: median, IQR, outliers
  • plot these as a vertical or horizontal box structure

The box can also be asymmetric β€œhalf plots” if needed…

Histograms and Box Plots

Note how the histogram that dwells upon the mean and standard deviation, whereas the boxplot focuses on the median and quartiles. The former uses the values of the Quant variable, whereas the latter uses their sequence number or ranks.

Box plots are often used for example in HR operations to understand Salary distributions across grades of employees. Marks of students in competitive exams are also declared using Quartiles.

Figure 2: Box Plot Definitions

Box plots can show skew in distributions. In such cases the β€œbottom” and the β€œlid” of the box may not be the same size!

(a) Box Plot and Skewness
(b) Density and Skewness
Figure 3: Box Plot Discussions

In the Figure 3 (a), we see the difference between boxplots that show symmetric and skewed distributions. The β€œlid” and the β€œbottom” of the box are not of similar width in distributions with significant skewness.

Compare these with the corresponding Figure 3 (b).

Case Study-1: gss_wages dataset

We will first look at Wage data from the General Social Survey (1974-2018) conducted in the USA, which is used to illustrate wage discrepancies by gender (while also considering respondent occupation, age, and education). This is available on Vincent Arel-Bundock’s superb repository of datasets. Let us read into R directly from the website.

wages <- read_csv("https://vincentarelbundock.github.io/Rdatasets/csv/stevedata/gss_wages.csv")

The data has automatically been read into the webr session, so you can continue on to the next code chunk!

Examine the Data

As per our Workflow, we will look at the data using all the three methods we have seen.

glimpse(wages)
Rows: 61,697
Columns: 12
$ rownames   <dbl> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, …
$ year       <dbl> 1974, 1974, 1974, 1974, 1974, 1974, 1974, 1974, 1974, 1974,…
$ realrinc   <dbl> 4935, 43178, NA, NA, 18505, 22206, 55515, NA, NA, 4935, NA,…
$ age        <dbl> 21, 41, 83, 69, 58, 30, 48, 67, 51, 54, 89, 71, 27, 30, 22,…
$ occ10      <dbl> 5620, 2040, NA, NA, 5820, 910, 230, 6355, 4720, 3940, 4810,…
$ occrecode  <chr> "Office and Administrative Support", "Professional", NA, NA…
$ prestg10   <dbl> 25, 66, NA, NA, 37, 45, 59, 49, 28, 38, 47, 45, 50, 29, 33,…
$ childs     <dbl> 0, 3, 2, 2, 0, 0, 2, 1, 2, 2, 3, 1, 4, 3, 0, 1, 2, 3, 4, 8,…
$ wrkstat    <chr> "School", "Full-Time", "Housekeeper", "Housekeeper", "Full-…
$ gender     <chr> "Male", "Male", "Female", "Female", "Female", "Male", "Male…
$ educcat    <chr> "High School", "Bachelor", "Less Than High School", "Less T…
$ maritalcat <chr> "Married", "Married", "Widowed", "Widowed", "Never Married"…
skim(wages)
Data summary
Name wages
Number of rows 61697
Number of columns 12
_______________________
Column type frequency:
character 5
numeric 7
________________________
Group variables None

Variable type: character

skim_variable n_missing complete_rate min max empty n_unique whitespace
occrecode 3561 0.94 5 37 0 11 0
wrkstat 21 1.00 5 23 0 8 0
gender 0 1.00 4 6 0 2 0
educcat 135 1.00 8 21 0 5 0
maritalcat 27 1.00 7 13 0 5 0

Variable type: numeric

skim_variable n_missing complete_rate mean sd p0 p25 p50 p75 p100 hist
rownames 0 1.00 30849.00 17810.53 1 15425 30849 46273 61697.0 β–‡β–‡β–‡β–‡β–‡
year 0 1.00 1996.07 12.79 1974 1985 1996 2006 2018.0 β–†β–‡β–‡β–‡β–‡
realrinc 23810 0.61 22326.36 28581.79 227 8156 16563 27171 480144.5 ▇▁▁▁▁
age 219 1.00 46.18 17.56 18 32 44 59 89.0 β–‡β–‡β–†β–…β–‚
occ10 3561 0.94 4695.77 2627.72 10 2710 4720 6230 9997.0 β–ƒβ–…β–‡β–‚β–ƒ
prestg10 4186 0.93 43.06 12.99 16 33 42 50 80.0 ▃▇▇▃▁
childs 189 1.00 1.92 1.76 0 0 2 3 8.0 ▇▇▂▁▁ 
inspect(wages)

categorical variables:  
        name     class levels     n missing
1  occrecode character     11 58136    3561
2    wrkstat character      8 61676      21
3     gender character      2 61697       0
4    educcat character      5 61562     135
5 maritalcat character      5 61670      27
                                   distribution
1 Professional (19%), Service (16.9%) ...      
2 Full-Time (49.4%), Housekeeper (15.1%) ...   
3 Female (56.1%), Male (43.9%)                 
4 High School (51.5%) ...                      
5 Married (51.7%), Never Married (21.8%) ...   

quantitative variables:  
      name   class  min    Q1 median    Q3      max         mean           sd
1 rownames numeric    1 15425  30849 46273  61697.0 30849.000000 17810.534116
2     year numeric 1974  1985   1996  2006   2018.0  1996.073715    12.794470
3 realrinc numeric  227  8156  16563 27171 480144.5 22326.359234 28581.794499
4      age numeric   18    32     44    59     89.0    46.176177    17.561065
5    occ10 numeric   10  2710   4720  6230   9997.0  4695.774081  2627.724076
6 prestg10 numeric   16    33     42    50     80.0    43.060701    12.987526
7   childs numeric    0     0      2     3      8.0     1.923457     1.763569
      n missing
1 61697       0
2 61697       0
3 37887   23810
4 61478     219
5 58136    3561
6 57511    4186
7 61508     189
Data Dictionary for the wages dataset

From the dataset documentation page, we note that:

  • This is a large dataset (61K rows), with 11 variables:
  • year(dbl): the survey year
  • realrinc(dbl): the respondent’s base income (in constant 1986 USD
  • age(dbl): the respondent’s age in years
  • occ10(dbl): respondent’s occupation code (2010)
  • occrecode(chr): recode of the occupation code into one of 11 main categories
  • prestg10(dbl): respondent’s occupational prestige score (2010)
  • childs(dbl): number of children (0-8)
  • wrkstat(chr): the work status of the respondent (full-time, part-time, temporarily not working, unemployed (laid off), retired, school, housekeeper, other). 8 levels. 
  • gender(chr): respondent’s gender (male or female). 2 levels.
  • educcat(chr): respondent’s degree level (Less Than High School, High School, Junior College, Bachelor, or Graduate). 5 levels.
  • maritalcat(chr): respondent’s marital status (Married, Widowed, Divorced, Separated, Never Married). 5 levels.
Business Insights based on wages dataset
  • Fair amount of missing data; however with 61K rows, we can for the present simply neglect the missing data.
  • Good mix of Qual and Quant variables
Hypothesis and Research Questions
  • The target variable for an experiment that resulted in this data might be the realinc variable, the resultant income of the individual. Which is numerical variable.
  • Research Questions:
    • What is the basic distribution of realrinc?
    • Is realrinc affected by gender?
    • By educcat? By maritalcat?
    • Is realrinc affected by child?
    • Do combinations of these factors have an effect on the target variable?

These should do for now! But we should make more questions when have seen some plots!

Data Munging

Since there are so many missing data in the target variable realinc and there is still enough data leftover, let us remove the rows containing missing data in that variable.

Important

NOTE: This is not advised at all as a general procedure!! Data is valuable and there are better ways to manage this problem!

wages_clean <- 
  wages %>% 
  tidyr::drop_na(realrinc) # choose column or leave blank to choose all

Plotting Box Plots

Question-1: What is the basic distribution of realrinc?

Question-1: What is the basic distribution of realrinc? 
# Set graph theme
theme_set(new = theme_custom())
#
wages_clean %>% 
  gf_boxplot(realrinc ~ "Income") %>% # Dummy X-axis "variable"
  gf_labs(title = "Plot 1A: Income has a skewed distribution",
          subtitle = "Many outliers on the high side")

# Set graph theme
theme_set(new = theme_custom())
#
wages_clean %>% 
  ggplot() + 
  geom_boxplot(aes(y = realrinc, x = "Income")) +  # Dummy X-axis "variable"
  labs(title = "Plot 1A: Income has a skewed distribution",
          subtitle = "Many outliers on the high side")

Business Insights-1

  • Income is a very skewed distribution, as might be expected.
  • Presence of many higher-side outliers is noted.

Question-2: Is realrinc affected by gender?

Question-2: Is realrinc affected by gender? 
# Set graph theme
theme_set(new = theme_custom())
#
wages_clean %>% 
  gf_boxplot(gender ~ realrinc) %>% 
  gf_labs(title = "Plot 2A: Income by Gender")
##
wages_clean %>% 
  gf_boxplot(gender ~ log10(realrinc)) %>% 
  gf_labs(title = "Plot 2B: Log(Income) by Gender")
##
wages_clean %>% 
  gf_boxplot(gender ~ realrinc, fill = ~ gender) %>% 
  gf_refine(scale_x_log10()) %>% 
  gf_labs(title = "Plot 2C: Income filled by Gender, log scale")
(a) Split by Gender
(b) With log income
(c) With log scale
Figure 4: Income by Gender 
# Set graph theme
theme_set(new = theme_custom())
#
wages_clean %>% 
  ggplot() + 
  geom_boxplot(aes(y = gender, x = realrinc)) +
  labs(title = "Plot 2A: Income by Gender")
##
wages_clean %>% 
  ggplot() + 
  geom_boxplot(aes(y = gender, x = log10(realrinc))) + 
  labs(title = "Plot 2B: Log(Income) by Gender")
##
wages_clean %>% 
  ggplot() + 
  geom_boxplot(aes(y = gender, x = realrinc, fill = gender)) +
  gf_refine(scale_x_log10()) +
  labs(title = "Plot 2C: Income filled by Gender, log scale")
(a) Split by Gender
(b) With log income
(c) With log scale
Figure 5: Income by Gender

Business Insights-2

  • Even when split by gender, realincome presents a skewed set of distributions.
  • The IQR for males is smaller than the IQR for females. There is less variation in the middle ranges of realrinc for men.
  • log10 transformation helps to view and understand the regions of low realrinc.
  • There are outliers on both sides, indicating that there may be many people who make very small amounts of money and large amounts of money in both genders.

Question-3: Is realrinc affected by educcat?

Question-3: Is realrinc affected by educcat? 
# Set graph theme
theme_set(new = theme_custom())
#
wages_clean %>% 
  gf_boxplot(educcat ~ realrinc) %>% 
  gf_labs(title = "Plot 3A: Income by Education Category")
##
wages_clean %>% 
  gf_boxplot(educcat ~ log10(realrinc)) %>% 
  gf_labs(title = "Plot 3B: Log(Income) by Education Category")
##
wages_clean %>% 
  gf_boxplot(reorder(educcat, realrinc, FUN = median) ~ log(realrinc), 
             fill = ~ educcat,
             alpha = 0.3) %>% 
  gf_labs(title = "Plot 3C: Log(Income) by Education Category, sorted") %>% gf_labs(x = "Log Income", y = "Education Category")
##
wages_clean %>% 
  gf_boxplot(reorder(educcat, realrinc, FUN = median) ~ realrinc, 
             fill = ~ educcat,
             alpha = 0.5) %>% 
  gf_refine(scale_x_log10()) %>% 
  gf_labs(title = "Plot 3D: Income by Education Category, sorted",
          subtitle = "Log Income Scale") %>% 
  gf_labs(x = "Income", y = "Education Category")
(a) Split by Education Category
(b) With log income
(c) With log scale
(d) Split by Education Category
Figure 6: Income by Education Category 
# Set graph theme
theme_set(new = theme_custom())
#
wages_clean %>% 
  ggplot() +
  geom_boxplot(aes(realrinc, educcat)) + # (x,y) format
  labs(title = "Plot 3A: Income by Education Category")
##
wages_clean %>% 
  ggplot() + 
  geom_boxplot(aes(log10(realrinc), educcat)) + 
  labs(title = "Plot 3B: Log(Income) by Education Category")
##
wages_clean %>% 
  ggplot() + 
  geom_boxplot(aes(log(realrinc), 
                   reorder(educcat, realrinc, FUN = median), 
                   fill = educcat),
               alpha = 0.3) + 
  labs(title = "Plot 3C: Log(Income) by Education Category, sorted",
       x = "Log Income", y = "Education Category")
##
wages_clean %>% 
  ggplot() + 
  geom_boxplot(aes(realrinc, 
                   reorder(educcat, realrinc, FUN = median), 
                   fill = educcat),
               alpha = 0.3)+ 
  scale_x_log10() + 
  labs(title = "Plot 3D: Income by Education Category, sorted",
       subtitle = "Log Income Scale",
       x = "Income", y = "Education Category")
(a) Split by Education Category
(b) With log income
(c) With log scale
(d) Split by Education Category
Figure 7: Income by Education Category 
wages_clean %>% 
  gf_boxplot(educcat ~ realrinc) %>% 
  gf_labs(title = "Plot 3A: Income by Education Category")

##
wages_clean %>% 
  gf_boxplot(educcat ~ log10(realrinc)) %>% 
  gf_labs(title = "Plot 3B: Log(Income) by Education Category")

##
wages_clean %>% 
  gf_boxplot(reorder(educcat, realrinc, FUN = median) ~ log(realrinc), 
             fill = ~ educcat,
             alpha = 0.3) %>% 
  gf_labs(title = "Plot 3C: Log(Income) by Education Category, sorted") %>% gf_labs(x = "Log Income", y = "Education Category")

##
wages_clean %>% 
  gf_boxplot(reorder(educcat, realrinc, FUN = median) ~ realrinc, 
             fill = ~ educcat,
             alpha = 0.5) %>% 
  gf_refine(scale_x_log10()) %>% 
  gf_labs(title = "Plot 3D: Income by Education Category, sorted",
          subtitle = "Log Income Scale") %>% 
  gf_labs(x = "Income", y = "Education Category")

Business Insights-3

  • realrinc rises with educcat, which is to be expected.
  • However, there are people with very low and very high income in all categories of educcat
  • Hence educcat alone may not be a good predictor for realrinc.

We can do similar work with the other Qual variables. Let us now see how we can use more than one Qual variable and answer the last hypothesis, Question 4.

Question-4: Is the target variable realrinc affected by combinations of Qual factors gender, educcat, maritalcat and childs?

Important

This is a rather complex question and could take us deep into Modelling. Ideally we ought to:
- take each Qual variable, explain its effect on the target variable
- remove that effect and model the remainder ( i.e. residual) with the next Qual variable
- Proceed in this way until we have a good model.
if we are going to do this manually. There are more modern Modelling Workflows, that can do things much faster and without such manual tweaking.

So will simply plot box plots showing effects on the target variable of combinations of Qual variables taken two at a time. (We will of course use facetted box plots!)

We will also drop NA values all around this time, to avoid seeing boxplots for undocumented categories.

Question-4: Is realrinc affected by combinations of factors? 
# Set graph theme
theme_set(new = theme_custom())
#
wages %>% 
  drop_na() %>% 
  gf_boxplot(reorder(educcat, realrinc) ~ log10(realrinc),
             fill = ~ educcat, 
             alpha = 0.5) %>% 
  gf_facet_wrap(vars(childs)) %>% 
  gf_refine(scale_fill_brewer(type = "qual", palette = "Dark2")) %>% 
  gf_labs(title = "Plot 4A: Log Income by Education Category and Family Size", x = "Log income", y = "No. of Children")
##
wages %>% 
  drop_na() %>% 
  mutate(childs = as_factor(childs)) %>% 
  gf_boxplot(childs ~ log10(realrinc),
             group = ~ childs,
             fill = ~ childs, 
             alpha = 0.5) %>% 
  gf_facet_wrap(~ gender) %>% 
  gf_refine(scale_fill_brewer(type = "qual", palette = "Set3")) %>% 
  gf_labs(title = "Plot 4B: Log Income by Gender and Family Size",
          x = "Log income",
          y = "No. of Children")
(a) Split by Education Category and Family Size
(b) Split by Gender and Family Size
Figure 8: Income and Other Qual Variables 
# Set graph theme
theme_set(new = theme_custom())
#
wages %>% 
  drop_na() %>%
  ggplot() + 
  geom_boxplot(aes(log10(realrinc), reorder(educcat, realrinc),
                   fill = educcat), # aes() closes here
               alpha = 0.5) + 
  facet_wrap(vars(childs)) +
  scale_fill_brewer(type = "qual", palette = "Dark2") + 
  labs(title = "Plot 4A: Log Income by Education Category and Family Size", x = "Log income", y = "No. of Children")
##
wages %>% 
  drop_na() %>% 
  mutate(childs = as_factor(childs)) %>% 
  ggplot() + 
  geom_boxplot(aes(log10(realrinc),childs,
                 group = childs,
                 fill = childs), # aes() closes here
             alpha = 0.5) +
  facet_wrap(vars(gender)) +
  scale_fill_brewer(type = "qual", palette = "Set3") + 
  labs(title = "Plot 4B: Log Income by Gender and Family Size",
          x = "Log income",
          y = "No. of Children")
(a) Split by Education Category and Family Size
(b) Split by Gender and Family Size
Figure 9: Income and Other Qual Variables

Business Insights-4

  • From Figure 8 (a), we see that realrinc increases with educcat, across (almost) all family sizes childs.
  • However, this trend breaks a little when family sizes childs is large, say >= 7. Be aware that the data observations for such large families may be sparse and this inference may not be necessarily valid.
  • From Figure 8 (b), we see that the effect of childs on realrinc is different for each gender! For females, the income steadily drops with the number of children, whereas for males it actually increases up to a certain family size before decreasing again.

Conclusion

  • Box Plots β€œdwell upon” medians and Quartiles
  • Box Plots can show distributions of a Quant variable over levels of a Qual variable
  • This allows a comparison of box plots side by side to visibly detect differences in median and IQR across such levels

Your Turn

Datasets

  1. Click on the Dataset Icon above, and unzip that archive. Try to make distribution plots with each of the three tools.
  2. A dataset from calmcode.io https://calmcode.io/datasets.html
  3. Old Faithful Data in R (Find it!)

inspect the dataset in each case and develop a set of Questions, that can be answered by appropriate stat measures, or by using a chart to show the distribution.

References

  1. See the scrolly animation for a histogram at this website: Exploring Histograms, an essay by Aran Lunzer and Amelia McNamara https://tinlizzie.org/histograms/?s=09
  2. Minimal R using mosaic.https://cran.r-project.org/web/packages/mosaic/vignettes/MinimalRgg.pdf
  3. Sebastian Sauer, Plotting multiple plots using purrr::map and ggplot
R Package Citations
Package Version Citation
ggridges 0.5.6 Wilke (2024)
NHANES 2.1.0 Pruim (2015)
TeachHist 0.2.1 Lange (2023)
TeachingDemos 2.13 Snow (2024)
visualize 4.5.0 Balamuta (2023)
Balamuta, James. 2023. visualize: Graph Probability Distributions with User Supplied Parameters and Statistics. https://CRAN.R-project.org/package=visualize.
Lange, Carsten. 2023. TeachHist: A Collection of Amended Histograms Designed for Teaching Statistics. https://CRAN.R-project.org/package=TeachHist.
Pruim, Randall. 2015. NHANES: Data from the US National Health and Nutrition Examination Study. https://CRAN.R-project.org/package=NHANES.
Snow, Greg. 2024. TeachingDemos: Demonstrations for Teaching and Learning. https://CRAN.R-project.org/package=TeachingDemos.
Wilke, Claus O. 2024. ggridges: Ridgeline Plots in β€œggplot2”. https://CRAN.R-project.org/package=ggridges.
Back to top

Citation

BibTeX citation:
@online{v.2024,
  author = {V., Arvind},
  title = {πŸ“Š {Boxplots:} {Plotting} {Groups}},
  date = {2024-06-24},
  url = {https://av-quarto.netlify.app/content/courses/Analytics/Descriptive/Modules/24-BoxPlots/},
  langid = {en},
  abstract = {Quant and Qual Variable Graphs and their Siblings}
}
For attribution, please cite this work as:
V., Arvind. 2024. β€œπŸ“Š Boxplots: Plotting Groups.” June 24, 2024. https://av-quarto.netlify.app/content/courses/Analytics/Descriptive/Modules/24-BoxPlots/.