lab_tidyverse.Rmd

---
title: "Tidy Work in Tidyverse"
subtitle: "R Programming Foundation for Life Scientists"
output:
  bookdown::html_document2:
    highlight: textmate
    toc: true
    toc_float:
      collapsed: true
      smooth_scroll: true
      print: false
    toc_depth: 4
    number_sections: true
    df_print: default
    code_folding: none
    self_contained: false
    keep_md: false
    encoding: 'UTF-8'
    css: "assets/lab.css"
    include:
      after_body: assets/footer-lab.html
---

```{r,child="assets/header-lab.Rmd"}
```

```{r,echo=FALSE,message=FALSE,warning=FALSE,results='hide'}
library(dplyr)
library(nycflights13)
```

# Introduction

Welcome to the hands-on workshop "Tidy Work in Tidyverse". Most of the things necessary to complete the tutorials and challenges were covered in the lecture. However, sometimes the tasks require that you check the docs or search online. Not all our solutions are optimal. Let us know if you can do better or solve things in a different way. If stuck, look at hints, next google and if still stuck, turn to TA. It is a lot of material, do not fee bad if you do not solve all tasks. Good luck!

# Pipes

- Rewrite the following code chunk as one pipe (`magrittr`):

```{r,eval=FALSE}
my_cars <- mtcars[, c(1:4, 7)]
my_cars <- my_cars[my_cars$disp > mean(my_cars$disp), ]
print(my_cars)
my_cars <- colMeans(my_cars)
```

```{r, accordion=TRUE, echo=TRUE, eval=FALSE}
my_cars <- mtcars %>%
  select(c(1:4, 7)) %>%
  filter(disp > mean(disp)) %T>%
  print() %>%
  colMeans()
```

- Rewrite the correlations below using pipes.

```{r,eval=FALSE}
cor(mtcars)
```

```{r, accordion=TRUE, echo=TRUE, eval=FALSE}
mtcars %>% cor()
```

```{r,eval=FALSE}
cor(mtcars$gear, mtcars$mpg)
```

```{r, accordion=TRUE, echo=TRUE, eval=FALSE}
mtcars %$% cor(gear, mpg)
```

# Tibbles

- Convert the `mtcars` dataset to a tibble `vehicles`.

```{r,accordion=TRUE}
vehicles <- mtcars %>% as_tibble()
```

- Select the number of cylinders (`cyl`) variable using:
    - the `[[index]]` accessor,
    - the `[[string]]` accessor,
    - the `$` accessor.
    
```{r,accordion=TRUE}
vehicles[['cyl']]
vehicles[[2]]
vehicles$cyl
```

- Do the same selection as above, but using pipe and placeholders (use all three ways of accessing a variable).

```{r,accordion=TRUE}
vehicles %T>%
  {print(.[['cyl']])} %T>%
  {print(.[[2]])} %>%
  .$cyl
```

- Print the tibble.

```{r,accordion=TRUE}
vehicles
```

- Print the 30 first rows of the tibble.

```{r,accordion=TRUE}
vehicles %>% head(n = 30)
```

- Change the default behaviour of printing a tibble so that at least 15 and at most 30 rows are printed.

```{r,accordion=TRUE}
options(tibble.print_min = 15, tibble.print_max = 30)
```

- Convert `vehicles` back to a `data.frame` called `automobiles`.

```{r,accordion=TRUE}
automobiles <- as.data.frame(vehicles)
```

Do you think tibbles are lazy? Try to create a tibble that tests whether *lazy evaluation* applies to tibbles too.

```{r eval=FALSE}
tibble(x = sample(1:10, size = 10, replace = T), y = log10(x))
```

# NYC flights Challenge

The `nycflights13` package contains information about all flights that departed from NYC (i.e., EWR, JFK and LGA) in 2013: 336,776 flights with 16 variables. To help understand what causes delays, it also includes a number of other useful datasets: weather, planes, airports, airlines. We will use it to train working with tibbles and `dplyr`.

## Selecting columns

- Load the `nycflights13` package (install if necessary)

```{r,accordion=TRUE,eval=FALSE}
install.packages('nycflights13')
library(nycflights13)
```

- Read about the data in the package docs

```{r,accordion=TRUE}
?nycflights13
```

- Inspect the `flights` tibble.

```{r,accordion=TRUE}
flights
```

- Select all columns but `carrier` and `arr_time`

```{r,accordion=TRUE}
flights %>% select(-carrier, -arr_time)
```

- Select `carrier`, `tailnum` and `origin`

```{r,accordion=TRUE}
flights %>% select(carrier, tailnum, origin)
```

- Hide columns from `day` through `carrier`

```{r,accordion=TRUE}
flights %>% select(-(day:carrier))
```

- Select all columns that have to do with `arr`_ival (hint: `?tidyselect`)

```{r,accordion=TRUE}
flights %>% select(contains('arr_'))
```

- Select columns based on a vector `v <- c("arr_time", "sched_arr_time", "arr_delay")`

```{r,accordion=TRUE}
v <- c("arr_time", "sched_arr_time", "arr_delay")
flights %>% select(v) # or
flights %>% select(one_of(v))
```

- Rename column `dest` to `destination` using `select()` and `rename()`. What is the difference between the two approaches?

```{r,accordion=TRUE}
flights %>% select(destination = dest) %>% head()
flights %>% rename(destination = dest) %>% head()
# select keeps only the renamed column while rename returns the whole dataset
# with the column renamed
```

## Filtering rows

- Filter only the flights that arrived ahead of schedule

```{r,accordion=TRUE}
flights %>% filter(arr_delay < 0)
```

- Filter the flights that had departure delay between 10 and 33

```{r,accordion=TRUE}
flights %>% filter(dep_delay >= 10, dep_delay <= 33) # or
flights %>% filter(between(dep_delay, 10, 33))
```

- Fish out all flights with unknown arrival time

```{r,accordion=TRUE}
flights %>% filter(is.na(arr_time))
```

- Retrieve rows 1234:1258 (hint: `?slice`)

```{r,accordion=TRUE}
flights %>% slice(1234:1258)
```

- Sample (`?sample_n()`) 3 random flights per day in March

```{r,accordion=TRUE}
flights %>% filter(month == 3) %>%
  group_by(day) %>%
  sample_n(3)
```

- Show 5 most departure-delayed flights in January per carrier

```{r,accordion=TRUE}
flights %>%
  filter(month == 1) %>%
  group_by(carrier) %>%
  top_n(5, dep_delay)
```

- How many unique routes exists?

```{r,accordion=TRUE}
flights %>%
  mutate(route=paste(origin,"-",dest)) %>%
  distinct(route,.keep_all=T) %>%
  nrow()
```

- Which is the most frequent route?

```{r,accordion=TRUE}
# JFK - LAX
flights %>%
  mutate(route=paste(origin,"-",dest)) %>% 
  group_by(route) %>%
  count() %>%
  arrange(-n)
```

## Trans(mutations)

- `air_time` is the amount of time in minutes spent in the air. Add a new column `air_spd` that will contain aircraft's airspeed in mph

```{r,accordion=TRUE}
flights %>% mutate(air_spd = distance/(air_time / 60))
```

- As above, but keep only the new `air_spd` variable

```{r,accordion=TRUE}
flights %>% transmute(air_spd = distance/(air_time / 60))
```

## Groups and counts

- Use `group_by()`, `summarise()` and `n()` to see how many planes were delayed (departure) every month

```{r,accordion=TRUE}
flights %>%
  filter(dep_delay > 0) %>%
  group_by(month) %>%
  summarise(num_dep_delayed = n())
```

- What was the mean `dep_delay` per month?

```{r,accordion=TRUE}
flights %>%
  group_by(month) %>%
  summarise(mean_dep_delay = mean(dep_delay, na.rm = T))
```

- Count the number of incoming delayed flights from each unique origin and sort origins by this count (descending)

```{r,accordion=TRUE}
flights %>%
  filter(arr_delay > 0) %>%
  group_by(origin) %>%
  summarise(cnt = n()) %>%
  arrange(desc(cnt))
```

- Use `summarise()` to sum total `dep_delay` per month in hours

```{r,accordion=TRUE}
 flights %>%
   group_by(month) %>%
   summarize(tot_dep_delay = sum(dep_delay/60, na.rm = T))
```

- Run `group_size()` on `carrier` what does it return?

```{r,accordion=TRUE}
flights %>%
    group_by(carrier) %>%
    group_size()
```

- Use `n_groups()` to check the number of unique origin-carrier pairs,

```{r,accordion=TRUE}
flights %>%
    group_by(carrier) %>%
    n_groups()
```

**Note on `ungroup`** Depending on the version of `dplyr` you may or may need to use the `ungroup()` if you want to group your data on some other variables. In the newer versions, `summarise` and `mutate` drop one aggregation level.