forked from bvkrauth/is4e
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy path11-Using-R.Rmd
More file actions
830 lines (666 loc) · 27.3 KB
/
Copy path11-Using-R.Rmd
File metadata and controls
830 lines (666 loc) · 27.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
# Using R {#using-r}
```{r setup11, include=FALSE}
knitr::opts_chunk$set(echo = TRUE,
prompt = FALSE,
tidy = FALSE,
collapse = TRUE)
library("tidyverse")
EmpData <- read_csv("sampledata/EmploymentData.csv")
```
In a previous chapter we learned some
[basic R terminology and how to run R](#an-introduction-to-r), and how to use R
to [read and view data](#reading-and-viewing-data-in-r). We are now ready to
start using R with real data.
This chapter will use R to clean and analyze the historical Canadian employment
data. We will also use ggplot, a powerful graphing system that is part of the
Tidyverse package.
::: {.goals data-latex=""}
**Chapter goals**
In this chapter, we will learn how to:
1. Use the pipe operator.
2. Use the `mutate` function to add or modify variables in a data table.
3. Use the `filter`, `select`, and `arrange` functions to modify a data table.
4. Calculate a univariate statistic in R.
5. Construct a table of summary statistics in R.
6. Recognize and handle missing data problems in R.
7. Construct a simple or binned frequency table in R.
8. Perform simple probability calculations in R.
9. Create a histogram with ggplot.
10. Create a line graph with ggplot.
:::
To prepare for this chapter, review the [introduction to R](#introduction-to-r).
Then open RStudio, load the Tidyverse, and read in our employment data:
```{r ReadEmpDataFromCSV_11, eval=FALSE}
library(tidyverse)
EmpData <- read_csv("https://bookdown.org/bkrauth/IS4E/sampledata/EmploymentData.csv")
```
If you prefer, you can download the employment data file from [https://bookdown.org/bkrauth/IS4E/sampledata/EmploymentData.csv](sampledata/EmploymentData.csv) and have R read it in locally:
```{r ReadEmpDataFromCSV_local, eval=FALSE}
library(tidyverse)
EmpData <- read_csv("EmploymentData.csv")
```
## Data cleaning in R {#data-cleaning-in-r}
We will not spend a lot of time on data cleaning in R, as we can always clean
data in Excel and export it to R. However, we have a few Tidyverse tools that
are useful to learn.
The Tidyverse includes four core functions for modifying data:
- `mutate()` allows us to add or change variables.
- `filter()` allows us to select particular observations, much like Excel's
filter tool.
- `arrange()` allows us to sort observations, much like Excel's sort tool.
- `select()` allows us to select particular variables.
All four functions follow a common syntax that is designed to work with a
convenient Tidyverse tool called the "pipe" operator.
### The pipe operator
The ***pipe operator*** is part of the Tidyverse and is written `%>%`. Recall
that an operator is just a symbol like `+` or `*` that performs some function
on whatever comes before it and whatever comes after it.
The pipe operator is unusual because it doesn't allow you to *do* anything new.
Instead, it makes your code easier to read. To see how it works, I will show you
a few examples.
:::example
**The pipe operator**
```{r PipeExamples}
# This is equivalent to names(EmpData)
EmpData %>% names()
# This is equivalent to sqrt(2)
2 %>% sqrt()
# This is equivalent to cat(sqrt(2)," is the square root of 2")
2 %>% sqrt() %>% cat(" is the square root of 2")
```
:::
As you can see from the examples, R's rule for interpreting the pipe operator is
that the object *before* the `%>%` is taken as the first argument for the
function *after* the `%>%`.
The pipe operator does not add any functionality to R; anything you can do with
it can also be done without it. But it addresses a common problem: we often
want to perform multiple transformations on a data set, but doing so in the
usual functional language can lead to code that is quite difficult to read. The
pipe operator can be used to create much more readable code, as we will see in
the examples below.
### Mutate
The most important data transformation function is ***mutate***, which allows us
to change or add variables.
:::example
**Using `mutate()` to modify a variable**
We will start by changing the **MonthYr** variable from character (text) to
date, using the `as.Date()` function:
```{r MutateChangeMonthYr}
# Change MonthYr to date format
EmpData %>%
mutate(MonthYr = as.Date(MonthYr, "%m/%d/%Y"))
```
As you can see, the **MonthYr** column is now labeled as a date rather than
text. Like Excel, R has an internal representation of dates that allows for
correct ordering and calculations, but displays dates in a standard
human-readable format.
:::
Mutate can be used to add variables as well as changing them.
:::example
**Using `mutate()` to add a variable**
Suppose we want to create versions of **UnempRate** and **LFPRate** that are
expressed in percentages rather than decimal units:
```{r MutateAddUnempPct}
# Add UnempPct and LFPPct
EmpData %>%
mutate(UnempPct = 100*UnempRate) %>%
mutate(LFPPct = 100*LFPRate)
```
If you look closely, you can see that the **UnempPct** and **LFPPct** variables
are now included in the data table.
:::
Before we go any further, note that `mutate()` is a function that *returns* a
*new* data table. In order to actually change the original data table, we will
need to use the assignment (`<-`) operator.
:::example
**Modifying a data table**
Take a look at `EmpData` data table.
```{r PrintEmpDataAgain}
print(EmpData)
```
As you can see, the **MonthYr** variable is still listed as a character variable
and the new **UnempPct** and **LFPPct** variables do not seem to exist.
In order to change `EmpData` itself, we need to assign that new object *back*
to `EmpData`.
```{r MutatePermanently}
# Make permanent changes to EmpData
EmpData <- EmpData %>%
mutate(MonthYr = as.Date(MonthYr, "%m/%d/%Y")) %>%
mutate(UnempPct = 100*UnempRate) %>%
mutate(LFPPct = 100*LFPRate)
```
We can confirm that now we have changed `EmpData`: **MonthYr** is a date
variable, and the variables **UnempPct** and **LFPPct** are part of the table.
```{r ConfirmPermanence}
print(EmpData)
```
:::
### Filter, select, and arrange
Excel provides a tool called "filtering" that allows you to choose observations
according to some criteria you specify. R has a function called `filter()`
that fills this purpose.
:::example
**Using `filter()` to choose observations**
Suppose we want to know more about the months in our data set with the *highest*
unemployment rates. We can use `filter()` for this purpose:
```{r FilterEmpData}
# This will give all of the observations with unemployment rates over 12.5%
EmpData %>%
filter(UnempPct > 12.5)
```
As you can see, only 8 of the 541 months in our data have unemployment rates
over 12.5% - the worst months of the 1982-83 recession, and April and May of
2020.
:::
Similarly, R has a function called `select()` that allows you to choose specific
variables.
:::example
**Using `select()` to choose variables**
Suppose that we only want to see a few pieces of information about the months
with the highest unemployment rates:
```{r SelectEmpData}
# This will take out all variables except a few
EmpData %>%
filter(UnempPct > 12.5) %>%
select(MonthYr, UnempRate, LFPPct, PrimeMinister)
```
:::
Finally, R has a function called `arrange()` that sorts observations according
to some criteria you specify.
:::example
**Using `arrange()` to sort observations**
Suppose that we want to show months in descending order by unemployment rate
(i.e., the highest unemployment rate first). We can use `arrange()` to sort
rows in this way:
```{r ArrangeEmpData}
# This will sort the rows by unemployment rate
EmpData %>%
filter(UnempPct > 12.5) %>%
select(MonthYr, UnempPct, LFPPct, PrimeMinister) %>%
arrange(UnempPct)
```
:::
Now I should probably say: these results imply nothing meaningful about the
economic policy of either Pierre or Justin Trudeau. The severe worldwide
recessions in 1982-83 (caused by US monetary policy) and 2020-2021 (caused
by the COVID-19 pandemic) were caused by world events largely outside the
control of Canadian policy makers.
:::example
**The usefulness of the pipe operator**
I mentioned earlier that the pipe operator is never strictly necessary, and that
it is always possible to write equivalent R code without it. But it does make
our code substantially more clear and readable.
```{r WithoutPipe, eval=FALSE}
# Here is some code using the pipe
EmpData %>%
filter(UnempPct > 12.5) %>%
select(MonthYr, UnempPct, LFPPct, PrimeMinister) %>%
arrange(UnempPct)
# This is what the same code looks like without the pipe
arrange(select(filter(EmpData,UnempPct>12.5),MonthYr,UnempPct,LFPPct,PrimeMinister),UnempPct)
```
:::
### Saving code and data
It is possible to save your data set in R's internal format just like you would
save an Excel file. But I'm not going to tell you how to do that, because what
you *really* need to do is save your code.
Because it is command-based, R enables an entirely different and much more
reproducible model for data cleaning and analysis. In Excel, the original data,
the data cleaning, the data analysis, and the results are all mixed in together
in a single file or even a single worksheet. This is convenient and simple to
use in many applications, but it can lead to disaster in more complex projects.
In contrast, R allows you to have three separate files or groups of files
1. Your **source data**
- Leave it exactly as you originally obtained it.
- Don't forget to document its provenance.
2. Your **code** to clean and analyze the data
- Maintain it carefully using master versions, working copies and archives.
- Your code can be saved in an R script, or as part of an R Markdown document.
- Your code can be split into multiple files, for example one file to clean
the data and another to analyze it.
- The code for my own research projects is often split into 10 or more
separate files, plus a "master" file that runs them all in the correct
order.
3. Your **cleaned data** and **results**
- Treat them as temporary output files.
- You should be able to regenerate them from the source data at any time by
just re-running your code.
Again, reproducibility means that we always have all of the information to
regenerate our results directly from the source data.
:::example
**Creating an R script to clean our data**
Create an R script called `EmpData.R` with the following code:
```{r fullScript, eval=FALSE}
# Load the Tidyverse
library(tidyverse)
# Load the data
EmpData <- read_csv("https://bookdown.org/bkrauth/IS4E/sampledata/EmploymentData.csv")
# Clean the data
EmpData <- EmpData %>%
mutate(MonthYr = as.Date(MonthYr, "%m/%d/%Y")) %>%
mutate(UnempPct = 100*UnempRate) %>%
mutate(LFPPct = 100*LFPRate)
```
:::
## Data analysis in R {#data-analysis-in-r}
Having read and cleaned our data set, we can now move on to some summary
statistics.
### The summary function
The `summary()` function will give a basic summary of any object. Exactly what
that summary looks like depends on the object. For tibbles, `summary()` produces
a set of summary statistics for each variable:
::: example
**Summarizing a data table**
```{r SummaryEmpData}
summary(EmpData)
```
:::
### Univariate statistics
The R function `mean()` calculates the sample average of any numeric vector.
There are also functions to calculate variance, standard deviation, and other
summary statistics.
::: example
**Univariate statistics for a single variable**
```{r MeanUnempPct}
# MEAN calculates the mean of a single variable
mean(EmpData$UnempPct)
# VAR calculates the sample variance
var(EmpData$UnempPct)
# SD calculates the standard deviation
sd(EmpData$UnempPct)
# MEDIAN calculates the sample median
median(EmpData$UnempPct)
```
:::
In real-world data, some variables have ***missing values*** for one or more
observations. Values can be missing for various reasons: information may be
unavailable, or the variable may not be applicable to the case. In R, missing
values are given the special value `NA` which stands for "not available."
::: example
**Missing values in the employment data**
The **AnnPopGrowth** variable in our data set is missing for the first year of
data (1976), since calculating the growth rate for 1976 would require data from
1975, and we don't have that data.
```{r ShowMissingValues}
EmpData %>%
select(MonthYr, Population, AnnPopGrowth) %>%
print(n=15) # n=15 means print the first 15 rows
```
:::
By default, math in R follows the [IEEE-754](https://en.wikipedia.org/wiki/IEEE_754)
standard for numerical arithmetic, which says that any calculation
involving `NA` should also result in `NA`.
::: example
**Calculations with `NA` produce `NA`**
```{r MeanAnnPopGrowth}
# Some values of AnnPopGrowth are NA, so its mean is also NA
mean(EmpData$AnnPopGrowth)
# The same goes for other statistics:
sd(EmpData$AnnPopGrowth)
var(EmpData$AnnPopGrowth)
median(EmpData$AnnPopGrowth)
```
:::
Some applications automatically drop missing data from the calculation. This is
a simple solution, but R does what it does for a good reason. Missing values
can be a sign of an error in the data or code, so we should always catch them
and investigate their cause before dropping them.
Once we have investigated the missing values, we can tell R explicitly to
exclude them from the calculation by adding the `na.rm = TRUE` optional
argument.
::: example
**Handling missing values**
In the case of our **AnnPopGrowth** variable, We know why it is missing: we
cannot calculate annual growth rates in the first year of our data. So we can
exclude the missing values.
```{r MeanAnnPopGrowthWithoutNA}
# We can add the na.rm = TRUE argument to drop the missing values
mean(EmpData$AnnPopGrowth, na.rm = TRUE)
# These lines of code will do the same thing:
mean(na.omit(EmpData$AnnPopGrowth))
mean(EmpData$AnnPopGrowth[!is.na(EmpData$AnnPopGrowth)])
```
:::
::: {.fyi data-latex=""}
**Missing values in Excel**
Excel also has rules for handling missing values. You can represent missing
values in one of two ways:
1. Leave cells with missing values blank, or enter some text string like "."
- Excel will *exclude* these cells from most statistical calculations,
- This is similar to how R handles `NA` values with the `na.rm=TRUE`
argument.
2. Indicate missing cells with the formula `=NA()`.
- The `NA()` function returns the error code `#N/A`, as will any function of
`NA()`
- This is similar to how R handles `NA` values by default.
:::
### Tables of statistics
Suppose we want to calculate the sample average for each column in our tibble.
We could just call `mean()` for each of them, but there is a quicker way.
:::example
**Producing a table of statistics**
Here is the code to do that:
```{r MeanTable}
# Mean of each column
EmpData %>%
select(where(is.numeric)) %>%
lapply(mean, na.rm = TRUE)
```
I would not expect you to come up with this code, but maybe it kind of makes
sense.
- The `select(where(is.numeric))` step selects only the columns in
`EmpData` that are numeric rather than text.
- The `lapply(mean,na.rm=TRUE)` step calculates `mean(x,na.rm=TRUE)`
for each (numeric) column `x` in `EmpData`.
We can use this method with any function that calculates a summary statistic:
```{r SDTable}
# Standard deviation of each column
EmpData %>%
select(where(is.numeric)) %>%
lapply(sd, na.rm = TRUE)
```
:::
### Frequency tables
We can construct simple frequency tables for discrete variables using the
`count()` function
::: example
**Using `count()` to make a simple frequency table**
```{r CountTables}
# COUNT creates a frequency table for discrete variables
EmpData %>%
count(PrimeMinister)
```
:::
We can also construct binned frequency tables for continuous variables using
the `count()` function in combination with the `cut_interval()` function.
::: example
**Using `count()` to make a binned frequency table**
```{r BinCountTables}
# COUNT and CUT_INTERVAL create a binned frequency table
EmpData %>%
count(cut_interval(UnempPct, 6))
```
:::
As you might imagine, there are various ways of customizing the intervals just
like in Excel.
### Probability distributions in R
Just like Excel, R has a family of built-in functions for each commonly-used
probability distribution.
::: example
**Some probability distributions**
The `dnorm()`, `pnorm()`, and `qnorm()` functions return PDF, CDF, and quantile
functions of the normal distribution, while the `rnorm()` function returns a
vector of normally distributed random numbers:
```{r}
# The N(0,1) PDF, evaluated at 1.96
dnorm(1.96)
# The N(1,4) PDF, evaluated at 1.96
dnorm(1.96,mean=1,sd=4)
# The N(0,1) CDF, evaluated at 1.96
pnorm(1.97)
# The 97.5 percentile of the N(0,1) CDF
qnorm(0.975)
# Four random numbers from the N(0,1) distribution
rnorm(4)
```
There is a similar set of functions available for the uniform distribution
(`dunif`, `punif`, `qunif`, `runif`), the binomial distribution (`dbinom`,
`pbinom`, `qbinom`,`rbinom`), and Student's T distribution (`dt`, `pt`, `qt`,
`rt`), along with many others.
:::
## Graphs with ggplot {#graphs-with-ggplot}
R has a built in plotting command called `plot()`, but the Tidyverse also
contains a much more powerful graphics package called[^1101]
***ggplot***. The ggplot package has capabilities well beyond what R's built-in
commands can do, or what Excel can do.
[^1101]: The package is technically called `ggplot2` since it is the second
version of `ggplot`. But everyone calls it "ggplot" anyway.
### Creating a graph
The `ggplot` function can be used to create a graph. It has a non-standard
syntax.
::: example
**Two ggplot graphs**
We can start by making a histogram of the unemployment rate, much like the one
we made earlier in Excel
```{r ggplotHistogram}
ggplot(data = EmpData,
mapping = aes(x = UnempPct)) +
geom_histogram()
```
We can also make a time series (line) graph, much like the one we made earlier
in Excel.
```{r ggplotTimeSeries}
ggplot(data = EmpData,
mapping = aes(x = MonthYr,
y = UnempPct)) +
geom_line()
```
:::
The `ggplot()` function has a non-standard syntax, so some explanation is
needed:
- The first expression/line calls `ggplot()` to set up the basic structure of
the graph:
- The `data` argument tells R which data set (tibble) will be used.
- the `mapping` argument describes the main ***aesthetics*** of the graph,
i.e., the relationship in the data we will be graphing.
- For the histogram, our aesthetic includes only one variable.
- For the line graph, our aesthetic includes two variables.
- The rest of the command is one or more statements separated by a `+` sign.
These are called ***geometries*** and are graph elements to be included in the
plot.
- The `geom_histogram()` geometry produces a histogram.
- The `geom_line()` geometry produces a line.
A single graph can include multiple aesthetics and geometries, as we will see
shortly.
### Modifying a graph
Like Excel, the basic ggplot graph gives us some useful information but we can
improve upon it in various ways.
#### Titles and labels
You can use `labs()` to add titles, subtitles, alt text, etc. You can also use
`xlab()` and `ylab()` to change the axis titles.
:::example
**Modifying titles and labels**
```{r ggplotTitles}
ggplot(data = EmpData,
aes(x = MonthYr,
y =UnempPct)) +
geom_line() +
labs(title = "Unemployment rate",
subtitle = "January 1976 - January 2021",
alt = "Time series of Canada unemployment rate, January 1976 - January 2021",
caption = "Source: Statistics Canada, Labour Force Survey",
tag = "Canada") +
xlab("") +
ylab("Unemployment rate, %")
```
:::
#### Color
You can change the color of any geometric element using the `col=` argument.
::: example
**Using colors**
```{r ggplotColor}
ggplot(data = EmpData,
aes(x = MonthYr,
y = UnempPct)) +
geom_line(col = "darkorange")
```
:::
Colors can be given in ordinary English (or local language) words like "red"
or "blue." There are hundreds of built in color names including "beige",
"hotpink" and four different shades of "chartreuse". You can get the full list
of built-in color names by executing the expression `colors()` in the command
window. You can also set custom colors using color codes in RGB or CMYK
format.
Some geometric elements, such as the bars in a histogram, also have a
***fill*** color.
:::example
**Setting the fill color***
```{r ggplotFill}
ggplot(data = EmpData,
aes(x = UnempPct)) +
geom_histogram(col = "darkorange",
fill = "navyblue")
```
As you can see, the `col=` argument sets the color for the exterior of each bar,
and the `fill=` argument sets the color for the interior.
:::
### Adding graph elements
We can include multiple geometries in the same graph.
::: example
**Including multiple lines**
We can include lines for both unemployment and labour force participation:
```{r ggplotTwoLines}
ggplot(data = EmpData,
aes(x = MonthYr,
y = UnempPct)) +
geom_line(col = "navyblue") +
geom_line(aes(y = LFPPct),
col = "darkorange")
```
A few things to note here:
- The third line gives `geom_line()` an aesthetics argument `aes(y=LFPPct)`. This
overrides the aesthetics in the first line.
- We have used color to differentiate the two lines, but there is no legend or
label to tell the reader which line is which. We will need to fix that.
- The vertical axis is labeled **UnempPct** which is the name of one of the
variables but not the other. We will need to fix that.
:::
When including multiple elements, we may need to add information clarify what
element describes. We can add a legend, but it is better (and friendlier to
the color-blind) to just label the lines. We can use the `geom_text` geometry
to do this:
:::example
**Using `geom_text()` to add text**
Label each line by adding a `geom_text()` geometry.
```{r ggplotAddText}
ggplot(data = EmpData,
aes(x = MonthYr)) +
geom_line(aes(y = UnempPct),
col = "navyblue") +
geom_text(x = as.Date("1/1/2000", "%m/%d/%Y"),
y = 15,
label = "Unemployment",
col = "navyblue") +
geom_line(aes(y = LFPPct),
col = "darkorange") +
geom_text(x = as.Date("1/1/2000", "%m/%d/%Y"),
y = 60,
label = "LFP",
col = "darkorange")
```
Note that we used color to reinforce the relationship between the label and
text, and we used the `x=` and `y=` arguments to place the text in a particular
location. You will usually need to try out a few locations to get something
that works.
:::
Finally, we can combine all of these elements.
::: example
**Fancy graphs**
The graphs below combine all of the features described above:
```{r ggplotFancyHistogram}
# Use RGB codes to exactly match the color scheme of this book
AstroBlue <- "#002D62"
AstroOrange <- "#EB6E1F"
# Fancy histogram
ggplot(data = EmpData,
aes(x = UnempPct)) +
geom_histogram(binwidth = 0.5,
fill = AstroBlue) +
geom_density() +
labs(title = "Unemployment rate",
subtitle = paste("January 1976 - January 2021 (",
nrow(EmpData),
" months)",
sep = "",
collapse = ""),
alt = "Histogram of Canada unemployment rate, January 1976 - January 2021",
caption = "Source: Statistics Canada, Labour Force Survey",
tag = "Canada") +
xlab("Unemployment rate, %") +
ylab("Count")
```
```{r ggplotFancyTimeSeries}
# Use RGB codes to exactly match color scheme of this book
AstroBlue <- "#002D62"
AstroOrange <- "#EB6E1F"
# Fancy time series plot
ggplot(data = EmpData,
aes(x = MonthYr)) +
geom_line(aes(y = UnempPct),
col = AstroBlue) +
geom_text(x = as.Date("1/1/2000", "%m/%d/%Y"),
y = 15,
label = "Unemployment", col=AstroBlue) +
geom_line(aes(y = LFPPct),
col = AstroOrange) +
geom_text(x = as.Date("1/1/2000", "%m/%d/%Y"),
y = 60,
label = "LFP",
col = AstroOrange) +
labs(title = "Unemployment and LFP rates",
subtitle = paste("January 1976 - January 2021 (",
nrow(EmpData),
" months)",
sep = "",
collapse = ""),
alt = "Time series of Canada unemployment and LFP, January 1976 - January 2021",
caption = "Source: Statistics Canada, Labour Force Survey",
tag = "Canada") +
xlab("") +
ylab("Percent")
```
:::
## Chapter review {-#review-using-r}
As we have seen, we can do many of the same things in Excel and R. R is
typically more difficult to use for simple analysis tasks, and there is nothing
wrong with using Excel when it is easier. But the usability gap gets smaller
with more complicated tasks, and there are many tasks where Excel doesn't do
everything that R can do. You should think of them as complementary tools, and
be comfortable using both.
In this chapter we learned how to use the pipe operator, the four key Tidyverse
functions for cleaning data, and the necessary tools for calculating univariate
statistics. In addition, we learned the basics of ggplot.
Our next applications of R will be for the analysis of relationships between
[multiple variables](#multivariate-data-analysis). The capabilities of ggplot
for constructing graphical elements like scatter plots and curve fits will be
well beyond what you can do in Excel, and will hopefully justify your investment
in learning R.
## Practice problems {-#problems-using-r}
Answers can be found in the [appendix](#answers-using-r).
**GOAL #1: Use the pipe operator**
**GOAL #2: Use the mutate function to add or change a variable**
**GOAL #3: Use the filter, select, and arrange functions to modify a data table**
1. Starting with the data table `EmpData`:
a. Add the numeric variable **Year** based on the existing variable **MonthYr**. The
formula for **Year** should be `format(MonthYr, "%Y")`.
b. Add the numeric variable **EmpRate**, which is the proportion of the population
(**Population**) that is employed (**Employed**), also called the employment
rate or employment-to-population ratio.
b. Drop all observations from years before 2010.
c. Drop all variables except **MonthYr**, **Year**, **EmpRate**, **UnempRate**,
and **AnnPopGrowth**.
d. Sort observations by **EmpRate**.
e. Give the resulting data table the name `PPData`.
**GOAL #4: Calculate a univariate statistic in R**
**GOAL #5: Construct a table of summary statistics in R**
**GOAL #6: Recognize and handle missing data problems in R**
2. Starting with the `PPData` data table you created in question (1) above:
a. Calculate and report the mean employment rate since 2010.
b. Calculate and report a table reporting the median for all variables in
`PPData`.
c. Did any variables in `PPData` have missing data? If so, how did you
decide to address it in your answer to (b), and why?
**GOAL #7: Construct a simple or binned frequency table in R**
3. Using the `PPData`data set, construct a frequency table of the employment
rate.
**GOAL #8: Perform simple probability calculations in R**
4. Calculate the following quantities in R:
a. The 45th percentile of the $N(4,6)$ distribution.
b. The 97.5 percentile of the $T_8$ distribution.
c. The value of the standard uniform CDF, evaluated at 0.75.
d. 5 random numbers from the $Binomial(10,0.5)$ distribution.
**GOAL #9: Create a histogram with ggplot**
5. Using the `PPData`data set, create a histogram of the employment rate.
**GOAL #10: Create a line graph with ggplot**
6. Using the `PPData` data set, create a time series graph of the employment
rate.