---
title: "R + tidyr (Data Wrangling)"
date: "`r format(Sys.time(),  '%d %B %Y')`"
author: "Mohamed Hassan"
output:
  html_notebook:
    theme: cerulean
    toc: true
    toc_depth: 3
editor_options:
  markdown:
    wrap: 72
---

# Workshop goals

By the end of this workshop you should be able to:

- understand **tidy data** and why it matters for data analysis
- clean and wrangle data using **dplyr** (filter, select, mutate, summarise)
- reshape data using **tidyr** (`pivot_longer()`, `pivot_wider()`, `separate()`)
- combine datasets using **joins** (`right_join()`, `left_join()`, `inner_join()`, `anti_join()`)
- handle real messy cases: missing values, wrong column types, messy IDs, duplicate keys


# Getting started

## Installing R and RStudio

- [R:](https://cran.rstudio.com/) https://cran.rstudio.com/
- [RStudio:](https://www.rstudio.com/products/rstudio/download/) https://www.rstudio.com/products/rstudio/download/


Working with R is primarily text-based. The basic mode of use for R is that the user types in a command in the R language and presses enter, and then R computes and displays the result.

![](r_gui.jpg)

### RStudio

We will be working in [RStudio](https://www.rstudio.com/products/rstudio/download/). This surrounds the *console*, where one enters commands and views the results, with various conveniences. In addition to the console, RStudio provides panels containing:

* A *text editor*, where R commands can be recorded for future reference.
* A history of commands that have been typed on the console.
* An "environment" panel with a list of *variables*, which contain values that R has been told to save from previous commands.
* A file manager.
* Help on the functions available in R (Using a question mark before the function)
* A panel to show plots (graphs).


![](Rstudio_layout.png)





## R script vs RMD (Markdowns and Notebooks)

* To start an R script click `ctrl + shift + N` or `cmd + shift + N` on mac  
You can run lines of code, but the output won't be saved    

* To start an RMD click `ctrl + B/N` or `cmd + B/N` on mac to initialize a notebook  
Allows you to run and visualize output in the file, easy to share with collaborators showing both the code and all results/plots  



This is called the main chunk, where is has all the settings that you would like to apply along the notebook
```{r setup}
knitr::opts_chunk$set(echo = T, results = "show")
options(width=80)
require("knitr")

## To change the directory in a notebook/markdown
# opts_knit$set(root.dir = "path/to/directory/")
```


To insert a chunk, use `ctrl + alt + I`, or `command + option + I`

# R Basics

## The console
Open RStudio, click on the "Console" panel, type `3+2` and press enter. R displays the result of the calculation. In this document, we will be showing such an interaction with R as below.
```{r}
3 + 2
```

`+` is called an operator. R has other operators for basic mathematical calculations:  
`-` for subtraction  
`*` for multiplication  
`/` forward slash for division  
`^` hat for "power"

`*` has higher precedence than `+`.  We can use brackets  `( )`, if necessary. Try `1+2*3` and `(1+2)*3`

```{r}
1+2*3
```

```{r}
(1+2)*3
```


Making lines of code according to styler, highlight the lines of code, click `ctrl + shift + A` or `cmd + shift + A`


## Comparisons

We use `>`, `<`, `>=`, `==`, `<=`, `!=` for comparisons. Note that for comparisons its always a double sign to be used. This returns a "logical" value of `TRUE` or `FALSE`.

```{r}
3 * 3 == 9
```

This literally means Is the left hand side equal to the right hand side? Is 3 times 3 equal to 9?  


## Variables

A variable is a name that stores a value. We create a variable by assigining a value to it using `<-` which points to the left assigning the value of the right to the one on the left

```{r}
weight_kg = 65
weight_kg <- 65
```


To print the value stored in a variable, print the variable
```{r}
weight_kg
```

Names of the variables should not contain special characters or spaces, should not start with a number. Dots are ok, unlike many other programming languages


To list all variables in an environment 
```{r}
ls()
## to remove a variable/object
rm(weight_kg)

## to remove some variable of a certain pattern (starting/ending/including ....)
rm(list = ls(pattern = ""))
```




Any line of code inside a chunk or in the console that has a `#` before, is a comment. Useful to document your code so its easy to remember why we used this command

To add a `#` at the beginning of the line for multiple lines, highlight all lines, and click `ctrl + shift + c` or `cmd + shift + c`

We can do arithemtic on the variable
```{r}
weight_kg <- 65
2.2 * weight_kg
```


## Vectors
A vector is a collection of elements (numbers, words,.. etc)
To be able to combine elements into a vector, we use the function c() which stands for combine or concatenate

```{r}
c(1,2,3,4)
```


```{r}
vec <- c(10,20,30,40)
vec + 1 # Here the arithmetic + 1 is applied on every element of the vector
```


We can also combine vectors
```{r}
vec + vec 
# In this case, the 1st element of vec is added to the 1st element of vec, 2nd with 2nd.. and so on
```

To check the length of the vector we use the function length()
```{r}
length(vec)
```

## Data types in R

Most common data types in R are mainly:  
* Numeric (could be double which is the default, or integer)  
* Character (String)  
* Logical (TRUE/FALSE)  

To check the type of a vector, we use typeof() function
```{r}
x <- 5
typeof(x)

## To create an integer add L to the number
x <- 5L
typeof(x)
print(x)
class(x)
```

For Characters, we use single or double quotes 

```{r}
"Hello World"
plain_txt = 'This is a string'
print(plain_txt)
typeof(plain_txt)
class(plain_txt)
```

To be able to look for and modify a pattern, the vector should be a string!

There are also categorical vectors in which elements can be one of several "categories/levels"
```{r}
factor(c("mutant", "wildtype", "mutant"), 
       levels = c("wildtype", "mutant"))
```

Factor is important for arrangement of data in a plot for example.



## Accessing elements of vectors (Indexing)
To access elements, we use [] with a number (index)
```{r}
vec[1]
vec[-1]
vec[c(1,3)]
vec[1:4]
vec[1] <- 6
```

We can use a vector instead of numbers to index a vector 
```{r}
myindex <- c(4,2,1,3)
vec[myindex]

## which is the same as
vec[c(4,2,1,3)]
```

## Matrix
2-D tabular data structure in which all elements are of the same type (typically numeric matrices, but could also be character or logical)
```{r}
# To create a matrix
matrix()
# Example:
mat <- matrix(1:12, nrow = 2, ncol = 6, byrow = F)
mat
typeof(mat)
class(mat)
```

```{r}
# To get the dimensions of a matrix
dim(mat)
## get only how many rows
nrow(mat)
## get only how many columns
ncol(mat)
## get column names
colnames(mat)
## Accessing elements of a matrix
mat[1,3] # row number, # col number]
# this retuns the element at row 1, column 3
```

## Data frame
Similar to a matrix, the difference is that the columns of a dataframe could be of different types (could combine numeric values, strings, logical)
The index of each row could be from 1 to length(dataframe) or could have a unique name (can be accessed/modified using rownames function)

## Functions
a function is followed by parentheses ()
mean(), sd(), print(), rep()
```{r}
# To create a function
name_of_function <- function(){ # inside parentheses what the function takes
  # what the function should do
}

## then call the function
name_of_function()
```


The function takes arguments (options), some are mandatory and some are optional.
Arguments can be supplied in order (positional argument), or by the argument name (better to use).
Clicking `tab` for autocompletion helps us remember what arguments a function could take

```{r}
calc <- function(num1, num2){
  sum = num1 + num2
  print(sum)
}

calc(num1 = 10, 5)
```



## Lists

Lists can contain different kinds of elements
```{r}
mylist <- list(num= 42, greeting="Hello, World")

# to access elements: ## double square bracket to access
mylist[[1]]

mylist$greeting

```

If you are not sure about the type of a variable, run class
```{r}
class(vec)
class(mat)
str(mylist)
```

### Applying a function on a list (very practical)
```{r}
list_nums <- list(
  a = c(1, 2, 3),
  b = c(10, 20, 30),
  c = c(5, 5, 5)
)

lapply(list_nums, mean)
```

### Applying an anonymous function (when we want to perform many steps over the list)
```{r}
list_nums_modified <- lapply(list_nums, function(element){
  ## first do this on each element
  element + 4
  ## then do this
  return(element)
})
```


```{r}
## Applying the mean function over a list
lapply(list_nums, function(elem) elem^2) ## returns a list
```


### sapply (simple apply)
for a simplified version
```{r}
sapply(list_nums, function(elem) elem^2) ## returns a simplified form (vector)
```



## Loops

```{r}
for (numb in list_nums){
  print(numb)
}
```

## If conditional
```{r}
list_nums <- c(1,2,3,6,7)
for(numb in list_nums){
  if(numb > 5){
    print(numb)}
    else print("This number is smaller than 5, so will not be printed")
}
```



# Tidy data


Tidy data rules:
1. Each variable = one column
2. Each observation = one row
3. Each type of entity = one table

Why it matters:
- easier plotting
- easier modelling
- fewer bugs
- compatible with tidyverse grammar



## Installing packages

To install a package from [CRAN](https://cran.r-project.org/), we use the function install.packages 
```{r}
install.packages("styler")
install.packages("tidyverse")
```

### From Github

[Github](https://github.com) is a web-based platform for version control, collaboration, and code sharing, built around the Git system. It allows users to track changes to files, manage contributions from multiple people, and publish open-source projects.  

GitHub is widely used for:

- Sharing R packages before they are published on CRAN

- Developing packages collaboratively using pull requests and issues

- Storing analysis projects (scripts, R Markdown, data, documentation)

- Reproducible workflows, since every change is tracked


Installing a package from Github `install_github("author/package")`


![](Github_screenshot_1.png)

```{r}
install.packages("devtools")

## To load a library
library(devtools)
devtools::install_github("githubusername/repository")
install_github("immunogenomics/presto")

## instead of loading a whole package, we can call a function from a package using the format:
package::function()
  
## function select
AnnotationDbi::select()
dplyr::select()
  
## Example
devtools::install_github("immunogenomics/presto")
```


### From Bioconductor
[Bioconductor](https://www.bioconductor.org/)


Bioconductor is an open-source project that provides a large collection of R packages for biological and biomedical data analysis, especially high-throughput sequencing, genomics, transcriptomics, and proteomics.

- Curated, peer-reviewed packages: Every package goes through a structured review for code quality, documentation, and reproducibility.

- Focus on life sciences: Tools for RNA-seq, ChIP-seq, single-cell RNA-seq, epigenomics, variant analysis, annotation, and more.

- Reproducible workflows using standardized data structures (e.g., SummarizedExperiment, SingleCellExperiment).

- Frequent release cycle (twice yearly), synchronized with R versions.

Bioconductor has become a central ecosystem for computational biology because it promotes reproducibility, integrates with modern R workflows, and provides consistent interfaces for complex biological datasets.



![](Bioconductor_screen1.png)

![](Bioconductor_screen2.png)  

```{r}
## First, to install Bioconductor
if (!require("BiocManager", quietly = TRUE))
    install.packages("BiocManager")
BiocManager::install(version = "3.22")

## Installing packages from Bioconductor
BiocManager::install("DESeq2")
```

```{r}
install.packages("tidyverse")
install.packages("janitor")
install.packages("lubridate")
```

## Loading libraries
```{r}
library(tidyverse)
library(tidyr)
library(dplyr)
library(stringr)
library(readr)
library(janitor)
library(lubridate)

# To load multiple libraries at once, we can use pacman package, p_load function
pacman::p_load(tidyverse, styler, dplyr, stringr, janitor, lubridate)

```



## Working with directories
```{r}
## To check the current directory
getwd()
## To change the working directory
setwd("Path/to/directory/")

## Remember to change the directory for the whole notebook, we use the main chunk
```

## Importing data
```{r}
## Importing an R data structure object (.RDS)
myfile <- readRDS(file = "path/to/file.RDS")

## CSV file
myfile <- utils::read.csv()

## More general function
myfile <- utils::read.delim(file = "/path/to/file.extenstion", 
                  header = T, sep = "\t") ## the file has a header with column names, and each observation is separated by a tab
myfile <- readxl::read_xlsx("/path/to/file/file.xlsx")
```


## Checking the content of a file
```{r}
head(myfile) ## shows the first 6 observations, can change n argument
tail(myfile) ## shows the last 6 observations, can change n argument
str(myfile) ## checks the structure of the file as a variable
dim(myfile) ## checks the dimensions of a file (returns number of rows, number of columns)
nrow(myfile) # checks the number of rows
ncol(myfile)# checks the number of columns

## checks the column names
colnames(myfile)
names(myfile)
```

## Data maniuplation

### Adding columns to a dataframe
```{r}
myfile$newcol <- myfile$col1 * myfile$col2
```

### Access columns by name
```{r}
# one column
myfile$columnname
## more than a column
myfile[, c("col1", "col4", "col2")]
myfile[, c(1,4,2)]
```

### Access a column by index
```{r}
myfile[,4] # the row is empty, which means show all rows for column 4
```

### Access rows
```{r}
df[4, ] ## gets the 4th row for all columns
df[4:10] ## from 4th to 10th
```

### Select rows by a condition (exact)
```{r}
myfile_subset <- subset(myfile, given_column == "something") ## subset rows that follow this condition
```


### Select rows by a condition (more than one category)
```{r}
myfile_subset <- subset(myfile, 
                        given_column %in% c("something", "something else")) ## subset rows that follow this condition
```


### Select rows by a condition (exclusion of exact)
```{r}
myfile_subset <- subset(myfile, given_column != "something") ## subset rows that follow this condition
```

### Select rows by a condition (exclusion of more than one category)

```{r}
myfile_subset <- subset(myfile, given_column != "something" & 
         given_column != "something else" &
         given_column != "another thing")
```

Using the `!` to negate 
```{r}
myfile_subset <- subset(myfile,
                        !(given_column %in% c("something", "something else")))
```



There is no equivalent operator for `%in%`, but you can create it!  

```{r}
"%out%" <- Negate("%in%")
```


```{r}
myfile_subset <- subset(myfile, 
                        given_column %out% c("something", "something else")) 
## subset rows that do not follow this condition
```


```{r}
# clean column names
# df <- janitor::clean_names(df)

# install.packages("tidytuesdayR")
# tuesdata <- tidytuesdayR::tt_load('2023-10-24')
# df <- tuesdata$patient_risk_profiles

df <- readxl::read_xlsx("tuesdata_df.xlsx")
head(df)
```

```{r}
df$`predicted risk of Sudden Vision Loss, with no eye pathology causes`
```


```{r}
df <- janitor::clean_names(df)
head(df)
```



# Using tidyverse
Used for data wrangling and maniuplation

When we load tidyverse, by default, dplyr, readr, stringr, tibble, tidyr, purrr, and ggplot2 are loaded
```{r}
library(tidyverse)
```

We will work with [care state](https://github.com/rfordatascience/tidytuesday/tree/main/data/2025/2025-04-08) dataset from tidy tuesday
```{r}
care_state <- readr::read_csv(
  "https://raw.githubusercontent.com/rfordatascience/tidytuesday/main/data/2025/2025-04-08/care_state.csv"
)

care_state %>% glimpse()

```


- *state* >> The two-letter code for the state (or territory, etc) where the hospital is located.  
- *condition* >>	The condition for which the patient was admitted. Six categories of conditions are included in the data.  
- *measure_id* >>	The ID of the thing being measured. Note that there are 22 unique IDs but only 21 unique names.  
- *measure_name* >>	The name of the thing being measured. Note that there are 22 unique IDs but only 21 unique names.  
- *score* >>	The score of the measure.  
- *footnote* 	>> 	Footnotes that apply to this measure: 
  -- 5 = "Results are not available for this reporting period.", 
  -- 25 = "State and national averages include Veterans Health Administration (VHA) hospital data.", 
  -- 26 = "State and national averages include Department of Defense (DoD) hospital data.".  
- *start_date* >>	date 	The date on which measurement began for this measure.  
- *end_date* >>	date 	The date on which measurement ended for this measure.  

Q1: How many columns are there in the dataset? What columns exist?   
Q2: How many observations(rows)?  
Q3: How many different states exist in the dataset ?  
Q4: What categories are there in the condition column?  
Q5: What is being measured in the "measure_name" column?  
Q6: How many observations with footnote 5, 25, and 26?  
Q7: Which years are there in the start and end date columns?  



```{r}
starwars <- dplyr::starwars
```

```{r}
head(starwars)
```


```{r}
typeof(starwars)
class(starwars)
```

```{r}
## Get the name of all columns
colnames(starwars)
# Or 
names(starwars)
```
# Functions on rows
## 1. Filtering

`filter(dataframe, condition)` like `filter(dataframe, column == "something")`  
  
`filter(dataframe, condition1, condition2)` like `filter(dataframe, column == "something", column == "something")`


**Why it matters in healthcare/bioinformatics:** you almost always filter:
- QC failures   
- non-target tissue / condition  
- outliers  
- incomplete cases  


```{r}
human_starwars <- filter(starwars, 
                         species == "Human")
head(human_starwars)
```


```{r}
filter(starwars,
       species == "Human", height > 180)
```

## Combining a lot of functions
```{r}
df <- nycflights23::flights
mean(pull(filter(filter(df, origin == "JFK"), carrier == "AA", !is.na(arr_delay)), arr_delay))
```

Using a pipe ` %>% `. Shortcut is `ctrl + shift + M` or `command + shift + M`
```{r}
df %>%
  filter(origin == "JFK") %>%
  filter(carrier == "AA") %>%
  filter(!is.na(arr_delay)) %>%
  pull(arr_delay) %>%
  mean()
```

Even nicer
```{r}
df %>%
  filter(origin == "JFK",
         carrier == "AA",
         !is.na(arr_delay)) %>%
  summarise(avg_delay = mean(arr_delay)) %>% 
  pull(avg_delay)
```

*NOTE*  
The function `pull` is a function that turns a column into a vector
For example if you have a column called "genes", using pull turns the column into a vector with all these genes


## Common patterns
```{r}
starwars %>% filter(is.na(mass))                # missing values
starwars %>% filter(!is.na(height))             # non-missing
starwars %>% filter(between(height, 150, 200))  # within a range
```


## 2. arrange() — sort rows

```{r}
starwars %>% 
  arrange(height)

starwars %>% 
  arrange(desc(height))
```



## 3. distinct() — unique rows

```{r}
starwars %>% 
  distinct(species, homeworld)
```

## 4. slice_max, slice_min, slice_head, and slice_tail

```{r}
DEGs <- readxl::read_xlsx("DEGs.xlsx")

DEGs %>% 
  mutate(pct_diff = pct.1 - pct.2) %>% 
  relocate(pct_diff, .after = pct.2) %>% 
  filter(p_val_adj < 0.05) %>% 
  group_by(cluster) %>% 
  slice_max(n = 10, order_by = avg_log2FC)
```


`slice_max` for each group, get the *top* n observations
`slice_min` for each group, get the *bottom* n observations 
`slice_head`  for each group, get the *first* n observations
`slice_tail`  for each group, get the *last* n observations

If not grouped 

`slice_max()` → get the top n rows overall based on a variable.
`slice_min()` → get the bottom n rows overall based on a variable.
`slice_head()` → get the first n rows in the dataset.
`slice_tail()` → get the last n rows in the dataset.


################################################################################
################################################################################

# Functions on columns

## 1. select(), rename(), relocate()

```{r}
starwars %>%
  select(name, height, species, mass) %>%
  rename("weight_kg" = "mass") %>%
  relocate(weight_kg, .after = height)
```

`rename(dataframe, newcolname = oldname)` renames an existing column name to another
Some R versions requires "" for column renaming


`relocate` function allows the new column is added in a specific position, default (last column)


## 2. Removing a column

The function `select` can be used to positively or negatively select columns (in any order)


```{r}
df %>% 
  select(day, month, dep_time)
```


We can use `select(df, -col1)` for one column or multiple columns like `select(df, -col1, -col2, -col3)` or even better use (any_of)

```{r}
cols_to_drop <- c("year", "carrier")

df %>% 
  select(-any_of(cols_to_drop))
```


```{r}
df %>% 
  select(starts_with("dep"))

df %>% 
  select(ends_with("time"))

df %>% 
  select(contains("del"))

df %>% 
  select(matches("(delay|time)$"))

```

```{r}
df %>% 
  select(where(is.numeric))

df %>% 
  select(where(is.character))
```


## 3. Turning a column into multiple
```{r}
df_time_hour <- df %>% select(time_hour)
head(df_time_hour)
```


The function `separate(dataframe, col = "column_to_split", into = c("piece1", "piece2"), sep = "separator"))`

```{r}
df_time_hour <-  df_time_hour %>% separate(sep = " ", 
                col = time_hour, 
                into = c("yearmonthday", "time"), 
                remove = T)
head(df_time_hour)
```

```{r}
df_time_hour <- df_time_hour %>% 
  separate(sep = "-", 
                col = yearmonthday, 
                into = c(NA, "month", "day"), 
                remove = T)
head(df_time_hour)
```
The same function can be applied on the time column, into hour, minute, seconds

```{r}
df_time_hour <- df_time_hour %>% 
  separate(sep = ":", 
                col = time, 
                into = c("hr", "mn", "sec"), 
                remove = T)
head(df_time_hour)
```
## 4. uniting columns with unite()

```{r}
df_time_hour %>%
  unite("newcol", month, day, hr, mn, sec, sep = "-", remove = FALSE)
```

## 5. mutate() — create new variables

Adding a column follows this structure `mutate(df, "newcolumn" = df$existingcolumn)` or  `mutate(df, newcolumn = df$existingcolumn)`

```{r}
starwars %>%
  mutate(height_m = height / 100, 
         bmi = mass / (height_m^2)) %>%
  select(name, height, mass, height_m, bmi)
```

```{r}
new_df <- nycflights23::flights %>% 
  mutate("dag" = day)

new_df <- new_df %>% 
  mutate("Day_Month_Year" = str_c(new_df$day, 
                                  new_df$month, 
                                  new_df$year, sep = "_")) %>%
  relocate(Day_Month_Year, .before = year)
head(new_df)
```

### case_when() — categorical variables

`mutate` with `case_when` can be used to add a column based on a condition of another column that follows a pattern.

mutate(NewCol = case_when(grepl("pattern", ExistingColumn) ~ 
"AddedText", 
TRUE ~ "AddedText2"))  

which means Assign "AddedText" if the pattern exists in the ExistingColumn, if not, assign "AddedText2"

```{r}
starwars %>%
  mutate(height_group = case_when(
    height < 150 ~ "short",
    height < 190 ~ "medium",
    TRUE ~ "tall")
    ) %>%
  count(height_group)
```

## 6.1 Grouping

`group_by` is used to group the dataframe by a certain column, where another function could be applied on each group. For instance, here we calculate the average arrival delay for each carrier

```{r}
df %>%
  group_by(carrier) %>%
  summarise(
    avg_arr_delay = mean(arr_delay, na.rm = TRUE),
    n_flights = n()
  )
```


## 6.2 group_by() + summarise()

```{r}
starwars %>%
  group_by(species) %>%
  summarise(
    n = n(),
    mean_height = mean(height, na.rm = TRUE)
  ) %>%
  arrange(desc(n))
```

## 7. across() — multi-column operations

```{r}
starwars %>%
  summarise(across(where(is.numeric), ~ mean(.x, na.rm = TRUE)))
```


# Merging/Joining dataframes

```{r}
set.seed(12345)
patient_table <- tibble(
  patient_id = paste0("P", str_pad(1:12, 3, pad = "0")),
  diagnosis = sample(c("control", "sepsis", "cancer"), 12, replace = TRUE),
  age = sample(30:90, 12, replace = TRUE),
  sex = sample(c("F", "M"), 12, replace = TRUE)
)

site_table <- tibble(
  patient_id = paste0("P", str_pad(sample(1:12, 10), 3, pad = "0")),
  hospital = sample(c("SiteA", "SiteB", "SiteC"), 10, replace = TRUE),
  country = sample(c("US", "DE", "FR"), 10, replace = TRUE)
)

patient_table
site_table

```

## left_join()

```{r}
patients_full <- patient_table %>%
  left_join(site_table, by = "patient_id")

patients_full
```

## Debug: who did NOT match?

```{r}
patient_table %>%
  anti_join(site_table, by = "patient_id")
```

```{r}
airlines_df <- nycflights23::airlines
head(airlines_df)
```


```{r}
flights_df <- nycflights23::flights
merged_df <- left_join(flights_df, airlines_df, by = "carrier") 

merged_df %>%  
  dplyr::rename(airlines_nm = name) %>% 
  relocate(airlines_nm, .after = carrier)
```

`left_join(x, y)`	All from x  
`right_join(x, y)`	All from y  
`inner_join(x, y)`	Only matching rows  
`full_join(x, y)`	All rows from both tables, filling the missing with NAs  




# Reshaping data (Wide vs Long data shape)
```{r}
set.seed(12345)

metadata <- tibble(
  sample_id = paste0("S", str_pad(1:24, 3, pad = "0")),
  patient_id = paste0("P", str_pad(sample(1:12, 24, replace = TRUE), 3, pad = "0")),
  tissue = sample(c("tumor", "normal"), 24, replace = TRUE),
  treatment = sample(c("drugA", "drugB", "placebo"), 24, replace = TRUE),
  sex = sample(c("F", "M"), 24, replace = TRUE),
  age = sample(30:85, 24, replace = TRUE),
  batch = sample(c("batch1", "batch2"), 24, replace = TRUE)
)

metadata

genes <- paste0("Gene", 
                str_pad(1:50, 4, 
                        pad = "0"))

counts <- matrix(
  rnbinom(50 * 24, mu = 80, size = 1),
  nrow = 50,
  ncol = 24,
  dimnames = list(genes, metadata$sample_id)
)

counts_df <- as_tibble(counts, rownames = "gene_id")
counts_df
```

Problem: This is "wide". Many analyses need "long".


## 1. pivot_longer(): wide -> long

```{r, results='hold'}
counts_long <- counts_df %>%
  pivot_longer(
    cols = starts_with("S"),
    names_to = "sample_id",
    values_to = "count"
  )

counts_long
```

Now each row is: gene × sample.


Now we join counts with metadata

```{r, results='hold'}
head(metadata)
```


```{r}
counts_annot <- counts_long %>%
  left_join(metadata, by = "sample_id")

counts_annot %>% glimpse()
```



Here we summarise expression by group

Example: mean counts by tissue per gene

```{r}
gene_summary <- counts_annot %>%
  group_by(gene_id, tissue) %>%
  summarise(mean_count = mean(count), .groups = "drop")

gene_summary
```

## 2. pivot_wider(): long -> wide

Create a gene × tissue table (2 columns: tumor/normal)

```{r}
gene_wide <- gene_summary %>%
  pivot_wider(
    names_from = tissue,
    values_from = mean_count
  )

gene_wide
```


## 3. Binding rows and columns

```{r}
visit_a <- tibble(
  id = c("P01","P02","P03"),
  visit = "baseline",
  crp = c(2.1, 5.4, 1.9)
)

visit_b <- tibble(
  id = c("P04","P05"),
  visit = "week4",
  crp = c(3.3, 2.8),
  wbc = c(6.1, 5.7)   # extra column not in visit_a
)

```


### 3.1 rbind vs bind_rows (stacking rows)
```{r}
rbind(visit_a, visit_b)
```


```{r}
bind_rows(visit_a, visit_b)
```
here `bind_rows` does not give an error, making sure rows are stacked, and missing columns are created and filled with NA (wbc is NA for baseline rows)


### 3.2 bind_cols() vs cbind() (gluing columns)

```{r}
patients <- tibble(
  id = c("P01","P02","P03"),
  sex = c("F","M","F")
)

labs <- tibble(
  crp = c(2.1, 5.4, 1.9),
  wbc = c(5.8, 6.2, 5.1)
)

```

```{r}
cbind(patients, labs)
```

```{r}
bind_cols(patients, labs)
```

```{r}
labs_short <- tibble(crp = c(2.1, 5.4))

cbind(patients, labs_short)
```

```{r}
bind_cols(patients, labs_short)
```

# Recap 

## 1. **Column-wise** functions 
### (modify or choose columns): 

- select()  

- rename()  

- relocate()  

### Create or modify columns  

- mutate()  

### Identify columns using tidyselect helpers:  
- starts_with(), ends_with(), contains(), matches(), where()


## 2. **Row-wise** functions (filter, slice, row operations)
### Keep or remove rows

- filter()

- slice(), slice_head(), slice_tail(), slice_min(), slice_max()

### Order rows

- arrange()


## 3. **Group-wise** functions 
### (operate within groups)

### Grouping changes how other functions behave:

group_by(), ungroup()

## 4. **Data-frame–wise** functions 
### (operate on whole table)

- bind_rows()

- bind_cols()

- left_join(), inner_join(), full_join(), anti_join()


### Reshape data:

pivot_longer() and pivot_wider()


## 5. **Summary** functions 
### (usually inside summarise())

mean(), median(), sum(), n(), n_distinct(), sd(), var()



# Good-to-know symbols names
()  parentheses  
[]  square brackets  
{}  curly braces  
*  asterisk  
&  ampersand  
^  hat  
/  forward slash  
\\  backslash  
''  single quotes  
""  double quotes  
-  hyphen (dash)  
_  underscore  
~  tilda  
`  backtick  
!  exclamation mark  
.  period  
,  comma  
:  colon  
;  semi-colon  
$  dollar sign  
#  sharp  
%  percentage  
  
