Data

Where does Data come from, and Why do we visualize it

Using web-R

This tutorial uses web-r that allows you to run all code within your browser, on all devices. Most code chunks herein are formatted in a tabbed structure ( like in an old-fashioned library) with duplicated code. The tabs in front have regular R code that will work when copy-pasted in your RStudio session. The tab “behind” has the web-R code that can work directly in your browser, and can be modified as well. The R code is also there to make sure you have original code to go back to, when you have made several modifications to the code on the web-r tabs and need to compare your code with the original!

Keyboard Shortcuts

• Run selected code using either:
  • macOS: ⌘ + ↩︎/Return
  • Windows/Linux: Ctrl + ↩︎/Enter
• Run the entire code by clicking the “Run code” button or pressing Shift+↩︎.

Click on any Picture to Zoom

All embedded figures are displayed full-screen when clicked.

“Difficulties strengthen the mind, as labor does the body.”

— Seneca

Setting up R Packages

library(tidyverse) # Data processing with tidy principles
library(mosaic) # Our go-to package for almost everything
library(ggformula) # Our plotting package

library(Lock5withR) # Some neat little datasets from a lovely textbook
library(kableExtra)

Plot Themes

# https://stackoverflow.com/questions/74491138/ggplot-custom-fonts-not-working-in-quarto

# Chunk options
knitr::opts_chunk$set(
  fig.width = 7,
  fig.asp = 0.618, # Golden Ratio
  # out.width = "80%",
  fig.align = "center"
)
### Ggplot Theme
### https://rpubs.com/mclaire19/ggplot2-custom-themes

theme_custom <- function() {
  font <- "Roboto Condensed" # assign font family up front

  theme_classic(base_size = 14) %+replace% # replace elements we want to change

    theme(
      panel.grid.minor = element_blank(), # strip minor gridlines
      text = element_text(family = font),
      # text elements
      plot.title = element_text( # title
        family = font, # set font family
        size = 20, # set font size
        face = "bold", # bold typeface
        hjust = 0, # left align
        # vjust = 2                #raise slightly
        margin = margin(0, 0, 10, 0)
      ),
      plot.subtitle = element_text( # subtitle
        family = font, # font family
        size = 14, # font size
        hjust = 0,
        margin = margin(2, 0, 5, 0)
      ),
      plot.caption = element_text( # caption
        family = font, # font family
        size = 8, # font size
        hjust = 1
      ), # right align

      axis.title = element_text( # axis titles
        family = font, # font family
        size = 10 # font size
      ),
      axis.text = element_text( # axis text
        family = font, # axis family
        size = 8
      ) # font size
    )
}

# Set graph theme
theme_set(new = theme_custom())
#

(a) Composition VIII

(b) Blue

Figure 1: Kandinsky: Abstract Paintings, or Data Visualizations?

Where does Data come from?

We will need to form a basic understanding of basic scientific enterprise. Let us look at the slides. (Also embedded below!)

View slides in full screen

Why Visualize?

• We can digest information more easily when it is pictorial
• Our Working Memories are both short-term and limited in capacity. So a picture abstracts the details and presents us with an overall summary, an insight, or a story that is both easy to recall and easy on retention.
• Data Viz includes shapes that carry strong cultural memories; and impressions for us. These cultural memories help us to use data viz in a universal way to appeal to a wide variety of audiences. (Do humans have a gene for geometry?¹);
• It helps sift facts and mere statements: for example:

Figure 2: Rape Capital

Figure 3: Data Reveals Crime

• Visuals are a good starting point to make hypotheses of what may be happening in the situation represented by the data

Why Analyze?

• Merely looking at visualizations may not necessarily tell us the true magnitude or significance of things.
• We need analytic methods or statistics to assure ourselves, or otherwise, of what we might suspect is happening
• These methods also help to remove human bias and ensure that we are speaking with the assurance that our problem deserves.
• Analysis uses numbers, or metrics, that allow us to crystallize our ambiguous words/guesses into quantities that can be calculated with.
• These metrics are calculable from our data, of course, but are not directly visible, despite often being intuitive

So both visuals and analytics.

What are Data Types?

 

Tidy Data

Each variable is a column; a column contains one kind of data. Each observation or case is a row.

How do we Spot Data Variable Types?

By asking questions! Shown below is a table of different kinds of questions you could use to query a dataset. The variable or variables that “answer” the question would be in the category indicated by the question.

No	Pronoun	Answer	Variable/Scale	Example	What Operations?
1	How Many / Much / Heavy? Few? Seldom? Often? When?	Quantities, with Scale and a Zero Value.Differences and Ratios /Products are meaningful.	Quantitative/Ratio	Length,Height,Temperature in Kelvin,Activity,Dose Amount,Reaction Rate,Flow Rate,Concentration,Pulse,Survival Rate	Correlation
2	How Many / Much / Heavy? Few? Seldom? Often? When?	Quantities with Scale. Differences are meaningful, but not products or ratios	Quantitative/Interval	pH,SAT score(200-800),Credit score(300-850),SAT score(200-800),Year of Starting College	Mean,Standard Deviation
3	How, What Kind, What Sort	A Manner / Method, Type or Attribute from a list, with list items in some " order" ( e.g. good, better, improved, best..)	Qualitative/Ordinal	Socioeconomic status (Low income, Middle income, High income),Education level (HighSchool, BS, MS, PhD),Satisfaction rating(Very much Dislike, Dislike, Neutral, Like, Very Much Like)	Median,Percentile
4	What, Who, Where, Whom, Which	Name, Place, Animal, Thing	Qualitative/Nominal	Name	Count no. of cases,Mode

As you go from Qualitative to Quantitative data types in the table, I hope you can detect a movement from fuzzy groups/categories to more and more crystallized numbers.

Type of Variables

Each variable/scale can be subjected to the operations of the previous group. In the words of S.S. Stevens

the basic operations needed to create each type of scale is cumulative: to an operation listed opposite a particular scale must be added all those operations preceding it.

Some Examples of Data Variables

Example 1: AllCountries

Base R

head(AllCountries, 5) %>% arrange(desc(Internet))

web-r

Questions

Q1. How many people in Andorra have internet access?
A1. This leads to the Internet variable, which is a Quantitative variable, a proportion.² The answer is \(70.5\%\).

Example 2:StudentSurveys

Base R

head(StudentSurvey, 5)

web-r

Questions

Q.1. What kind of students are these?
A.1. The variables Gender, and Year both answer to this Question. And they are both Qualitative/Categorical variables, of course.
Q.2. What is their status in their respective families?
A.2. Hmm…they are either first-born, or second-born, or third…etc. While this is recorded as a number, it is still a Qualitative variable³! Think! Can you do math operations with BirthOrder? Like mean or median?
Q.3.How big are the families?
A.3. Clearly, the variable that answers is Siblings and since the question is synonymous with “how many”, this is a Quantitative variable.

What is a Data Visualization?

Data Viz = Data + Geometry

Shapes

Data Visualization is the act of “mapping” a geometric aspect/aesthetic to a variable in data. The aesthetic then varies in accordance with the data variable, creating (part of) a chart.

What might be the geometric aesthetics available to us? An aesthetic is a geometric property, such as x-coordinate, y-coordinate, length/breadth/height,radius,shape,size, linewidth, linetype, and even colour…

Common Geometric Aesthetics in Charts

Mapping

What does this “mapping” mean? That the geometric aesthetics are used to represent qualitative or quantitative variables from your data, by varying in accordance to the data variable.

For instance, length or height of a bar can be made proportional to theage or income of a person. Colour of points can be mapped to gender, with a unique colour for each gender. Position along an axis x can vary in accordance with a height variable and position along the y axis can vary with a bodyWeight variable.

Data Vis using R

A chart may use more than one aesthetic: position, shape, colour, height and angle,pattern or texture to name several. Usually, each aesthetic is mapped to just one variable to ensure there is no cognitive error. There is of course a choice and you should be able to map any kind of variable to any geometric aspect/aesthetic that may be available.

A Natural Mapping

Note that here is also a “natural” mapping between aesthetic and [kind of variable] Section 7, Quantitative or Qualitative. For instance, shape is rarely mapped to a Quantitative variable; we understand this because the nature of variation between the Quantitative variable and the shape aesthetic is not similar (i.e. not continuous). Bad choices may lead to bad, or worse, misleading charts!

In the above chart, it is pretty clear what kind of variable is plotted on the x-axis and the y-axis. What about colour? Could this be considered a z-axis in the chart? There are also other aspects that you can choose (not explicitly shown here) such as the plot theme(colours, fonts, backgrounds etc), which may not be mapped to data, but are nonetheless choices to be made. We will get acquainted with this aspect as we build charts.

Some essential concepts to master when working with charts in R are:

• geoms & aesthetics,
• scales,
• statistical transformations,
• coordinate transformations.
• the group aesthetic,
• position adjustments,
• facets,
• themes.

Basic Types of Charts

We can think of simple visualizations as combinations of aesthetics, mapped to combinations of variables. Some examples:

Geometries , Combinations, and Graphs

Variable #1	Variable #2	Chart Names	Chart Shape
Quant	None	Histogram and Density
Qual	None	Bar Chart
Quant	Quant	Scatter Plot, Line Chart, Bubble Plot, Area Chart
Quant	Qual	Pie Chart, Donut Chart, Column Chart, Box-Whisker Plot, Radar Chart, Bump Chart, Tree Diagram
Qual	Qual	Stacked Bar Chart, Mosaic Chart, Sankey, Chord Diagram, Network Diagram

Conclusion

Let us take a look at Wickham and Grolemund’s Data Science workflow picture:

Figure 4: Data Science Workflow

So there we have it:

• We import and clean the data
• Questions lead us to identify Types of Variables (Quant and Qual)
  
• Sometimes we may need to transform the data (long to wide, summarize, create new variables…)
• Further Questions lead to relationships between variables, which we describe using Data Visualizations
  
• Visualizations may lead to Hypotheses, which we Analyze or Model
• Data Visualizations are Data mapped onto Geometry 
• Multiple Variable-to-Geometry Mappings = A Complete Data Visualization
  
• Which is finally Communicated

You might think of all these Questions, Answers, Mapping as being equivalent to metaphors as a language in itself. And indeed, in R we use a philosophy called the Grammar of Graphics! We will use this grammar in the R graphics packages that we will encounter. Other parts of the Workflow (Transformation, Analysis and Modelling) are also following similar grammars, as we shall see.

AI Generated Summary and Podcast

This is a tutorial on data visualization using the R programming language. It introduces concepts such as data types, variables, and visualization techniques. The tutorial utilizes metaphors to explain these concepts, emphasizing the use of geometric aesthetics to represent data. It also highlights the importance of both visual and analytic approaches in understanding data. The tutorial then demonstrates basic chart types, including histograms, scatterplots, and bar charts, and discusses the “Grammar of Graphics” philosophy that guides data visualization in R. The text concludes with a workflow diagram for data science, emphasizing the iterative process of data import, cleaning, transformation, visualization, hypothesis generation, analysis, and communication.

References

1. Randomized Trials:
   

1. Martyn Shuttleworth, Lyndsay T Wilson (Jun 26, 2009). What is the Scientific Method? Retrieved Mar 12, 2024 from Explorable.com: https://explorable.com/what-is-the-scientific-method

2. Adam E.M. Eltorai, Jeffrey A. Bakal, Paige C. Newell, Adena J. Osband (editors). (March 22, 2023) Translational Surgery: Handbook for Designing and Conducting Clinical and Translational Research.

   • Part III. Clinical: fundamentals
   • Part IV: Statistical principles

A very lucid and easily explained set of chapters. ( I have a copy. Yes.)

1. https://safetyculture.com/topics/design-of-experiments/

2. Open Intro Stats: Types of Variables

3. Lock, Lock, Lock, Lock, and Lock. Statistics: Unlocking the Power of Data, Third Edition, Wiley, 2021. https://www.wiley.com/en-br/Statistics:+Unlocking+the+Power+of+Data,+3rd+Edition-p-9781119674160)

4. Claus Wilke. Fundamentals of Data Visualization. https://clauswilke.com/dataviz/

5. Tim C. Hesterberg (2015). What Teachers Should Know About the Bootstrap: Resampling in the Undergraduate Statistics Curriculum, The American Statistician, 69:4, 371-386, DOI:10.1080/00031305.2015.1089789. PDF here

6. Albert Rapp. Adding images to ggplot. https://albert-rapp.de/posts/ggplot2-tips/27_images/27_images

R Package Citations

Package	Version	Citation
ggformula	0.12.0	Kaplan and Pruim (2023)
Lock5withR	1.2.2	Pruim (2015)
mosaic	1.9.1	Pruim, Kaplan, and Horton (2017)
TeachingDemos	2.13	Snow (2024)

Kaplan, Daniel, and Randall Pruim. 2023. ggformula: Formula Interface to the Grammar of Graphics. https://CRAN.R-project.org/package=ggformula.

Pruim, Randall. 2015. Lock5withR: Datasets for “Statistics: Unlocking the Power of Data”. https://github.com/rpruim/Lock5withR.

Pruim, Randall, Daniel T Kaplan, and Nicholas J Horton. 2017. “The Mosaic Package: Helping Students to ‘Think with Data’ Using r.” The R Journal 9 (1): 77–102. https://journal.r-project.org/archive/2017/RJ-2017-024/index.html.

Snow, Greg. 2024. TeachingDemos: Demonstrations for Teaching and Learning. https://CRAN.R-project.org/package=TeachingDemos.

Footnotes

1. https://www.xcode.in/genes-and-personality/how-genes-influence-your-math-ability/

2. How might this data have been obtained? By asking people in a survey and getting Yes/No answers!

3. Qualitative variables are called Factor variables in R, and are stored, internally, as numeric variables together with their levels. The actual values of the numeric variable are 1, 2, and so on.