computer says no - debug youR code

class: center, middle

![:scale 30%](/assets/images/coding_club_logo_1.png)

<!-- Adjust the presentation to the session. Focus on the challenges,
 this is not a coding tutorial.

Note, to include figures, store the image in the `/docs/assets/images/yyyymmdd/`
    folder and use the jekyll base.url reference as done in this template
    or see https://jekyllrb.com/docs/liquid/tags/#links.
    using the scale attribute ![:scale 30%](...), you can adjust the image size.
-->

# 25 MAY 2023

## INBO coding club

Herman Teirlinck Building

01.23 - Léon Stynen

---
class: left, top

# Reminders

1. Did we confirm the room via _roomie_?
2. Did we start the recording?

---
class: left, top

# It's all about colors!

Last month we very briefly spoke about [cols4all](https://github.com/mtennekes/cols4all), an interesting package with a User Interface to choose your best color palette. Well, there is another very interesting color dedicated R package maybe you would like to give a try: [colorspace](https://colorspace.r-forge.r-project.org/index.html).

![:scale 90%](/assets/images/20230525/20230525_colorspace_homepage.png)

---
class: center, middle

![:scale 90%](/assets/images/20230525/20230525_badge_debug.png)

---
class: center, middle

![:scale 100%](/assets/images/20230525/20230525_debugging_phdcomics.gif)

Your 1 minute infinite joy after tons of frustration

---
class: center, middle

_Finding your bug is a process of confirming the many things that you believe are true — until you find one which is not true. _ (Norm Matloff)

![:scale 100%](/assets/images/20230525/20230525_intro_debugging.png)

---
class: left, top

## How to get started?

Check the [Each session setup](https://inbo.github.io/coding-club/gettingstarted.html#each-session-setup) to get started.

## First time coding club?

Check the [First time setup](https://inbo.github.io/coding-club/gettingstarted.html#first-time-setup) section to setup.

---
class: left, top

![:scale 100%](/assets/images/coding_club_sticky_concept.png)

---
class: center, top

# Share your code during the coding session

Go to https://hackmd.io/zkAmzjw1RnOuLvVRdtCH2g?edit and start by adding your name in section "Participants".

---
class: left, top

# Download data and code

You can download the material of today:

- automatically via `inborutils::setup_codingclub_session()`*

- manually** from GitHub folders [coding-club/data/20230525](https://github.com/inbo/coding-club/tree/master/data/20230525) and [coding-club/src/20230525](https://github.com/inbo/coding-club/tree/master/src/20230525)

__\* Note__: you can use the date in "YYYYMMDD" format to download the coding club material of a specific day, e.g. run `setup_codingclub_session("20220428")` to download the coding club material of April, 28 2022. If date is omitted, i.e. `setup_codingclub_session()`, the date of today is used. For all options, check the [tutorial online](https://inbo.github.io/tutorials/tutorials/r_setup_codingclub_session/).
 
 __\*\* Note__: check the getting started instructions on [how to download a single file](https://inbo.github.io/coding-club/gettingstarted.html#each-session-setup)

---
class: left, top

# Data and scripts description

- `20230525_eel_Anna.txt`, `20230525_eel_Amber.txt` and `20230525_eel_Lisa.txt`: eels tagging data. Thanks, Pieterjan Verhelst and LifeWatch Flanders for providing us these data!

- `20230525_challenges.R`: R script with code to start with

---
class: left, top

# Load libraries

```r
library(tidyverse)
```

---
class: left, top

# Debugging first rule

Read carefully the error message and search on the web: Google! Nowadays, you would maybe give a try to Artificial Intelligence tools as [ChatGPT](https://chat.openai.com/).

And let's train our eyes!

Example 1

```
Error in `mutate()`:
ℹ In argument: `delta_t = as.numeric(as.duration(time_first_dist_to_use -
  departure))`.
Caused by error in `as.duration()`:
! could not find function "as.duration"
Run `rlang::last_trace()` to see where the error occurred.
```

https://www.awesomescreenshot.com/image/40093399?key=8b0ede621e12a0f58eaf7c84fcb83d45

---
class: left, top

# Debugging first rule

Read carefully the error message and search on the web: Google! Nowadays, you would maybe give a try to Artificial Intelligence tools as [ChatGPT](https://chat.openai.com/).

And let's train our eyes!

Example 2

```
my_function <- function() {
 txt <- "fantastic"
 paste("R is" txt)
}
Error: unexpected symbol in:
" txt <- "fantastic"
 paste("R is" txt"
```
https://www.awesomescreenshot.com/image/40096422?key=151ba2b88885ab651c003414c0907886

---
class: left, top

# Debugging first rule

Read carefully the error message and search on the web: Google! Nowadays, you would maybe give a try to Artificial Intelligence tools as [ChatGPT](https://chat.openai.com/).

And let's train our eyes!

Example 3: error is far from being informative for "normal" people

```
my_gamma_est <- function(data) {
 # fit a gamma distribution to a vector of numbers
 m < mean(data)
 v <- var(data)
 s <- v/m
 a <- m/s
 return(list(a=a,s=s))
}
```

```
my_gamma_est(cats$Hwt)
Error in my_gamma_est(cats$Hwt) :
  'list' object cannot be coerced to type 'double'
```

As you can see in the screenshot linked here below, it seems ChatGPT cannot help us so much:

https://www.awesomescreenshot.com/image/40098063?key=2fcb63ab83df27bbb5b9997c0c418581

---
class: left, top

You have some code, it doesn't work as expected...
<center>![:scale 100%](/assets/images/20230525/20230525_print_in_action1.png)</center>

You start to search by inserting `print()` instructions here and there..

---
class: left, top

Welcome to the `print()` FESTIVAL!

<center>![:scale 100%](/assets/images/20230525/20230525_print_in_action2.png)</center>

And what if the error is not in your code but in a function embedded in it?

---
class: left, top

## Why debugging?

- save time: "debugging" using `print()` is time consumming  + you have to clean up your code after
- jump in **loops**, in **if-else** statements, in **functions**

---
class: center, middle

![:scale 90%](/assets/images/20230525/20230525_RStudio_IDE_cheatsheet.png)
[Download cheatsheet here](https://github.com/inbo/coding-club/blob/master/cheat_sheets/20230525_cheat_sheet_rstudio-ide.pdf)

---
class: center, middle

![:scale 90%](/assets/images/20230525/20230525_debug_mode_zoom_cheatsheet.png)

---
class: center, middle

![:scale 45%](/assets/images/20230525/20230525_debug_mode_zoom_cheatsheet2.png)

---
class: left, top

#### A live code intro

```
library(MASS) # provides `cats` data
gamma_est <- function(data) {
 # this fits a gamma distribution to a collection of numbers
 m <- mean(data)
 v <- var(data)
 s <- v/m
 a <- m/s
 return(list(a=a,s=s))
}
calc_var <- function(estimates){
 var_of_ests <- apply(estimates, 2, var)
 return(((n-1)^2/n)*var_of_ests)
}
gamma_jackknife <- function(data) {
 ## jackknife the estimation
 n <- length(data)
 jack_estimates <- gamma_est(data[-1])
 for (omitted_point in 2:n) {
 jack_estimates <- rbind(jack_estimates, gamma_est(data[-omitted_point]))
 }
 jack_var = calc_var(jack_estimates)
 return(sqrt(jack_var))
}
# jackknife gamma dist. estimates of cat heart weights
gamma_jackknife(cats$Hwt)
Error in FUN(newX[, i], ...) : is.atomic(x) is not TRUE
```

Credits: Chris Paciorek. Session 2.5 of his very interesting [tutorial](https://htmlpreview.github.io/?https://github.com/berkeley-scf/tutorial-R-debugging/blob/master/R-debugging.html)

---
class: left, top

# A memory aid

## Traceback your code

`traceback` your code to find where it stops to work.

In RStudio: in **Debug pane** -> **On Error** -> **Error Inspector** and then click on **Show Traceback** in the error message 
![:scale 100%](/assets/images/20230525/20230525_livecoding_show_traceback.png)
In basic R:
```r
options(error = "traceback") # trace the error back, better than options(error = "message")
```

---
class: left, top

# A memory aid

## Debug your code: activate debugging mode

### 1st alternative

Source the file with the functions only and set breakpoints (red circles) where you think something wrong happens

![:scale 50%](/assets/images/20230525/20230525_livecoding_show_breakpoint.png)

---
class: left, top

# A memory aid

## Debug your code: activate debugging mode

### 2nd alternative

Set `options(error = recover)` and select the function:

![:scale 100%](/assets/images/20230525/20230525_livecoding_show_error_recovery.png)

---
class: left, top

# A memory aid

## Debug your code: activate debugging mode

### 3rd alternative

Activate debugging mode on a specific function by using `debug(function_name)` or `debugonce(function_name)`.

```r
debug(calc_var) # or debugonce(calc_var)
```

Once this function is debugged, run `undebug(function_name)`.

---
class: left, top

# A memory aid

## Debug your code: work in debugging mode

Once you are in debugging mode, use **Next** (n) to move to next line, **Continue** (c) to go to next breakpoint (red circles)

![:scale 70%](/assets/images/20230525/20230525_livecoding_show_debugging_mode_actions.png)

---
class: left, top

# 123 magic

Write down a positive integer on a piece of paper:

```
886328712442992
```

Count up the number of even and odd digits, and the total number of digits:

```
10 5 15
```

String the digits of those three numbers together to make a new number:

```
10515
```

Perform the same operation on the obtained number:

```
1 4 5 ⟶ 145
```

And keep iterating:

```
1 2 3 ⟶ 123
```

You'll always arrive at the number 123!

Credits (login via Google account or some other methods): 
(EN) https://dodona.ugent.be/en/courses/239/series/2168/activities/192047393/ 
(NL) https://dodona.ugent.be/nl/courses/239/series/2168/activities/192047393/

---
background-image: url(/assets/images/background_challenge_1.png)
class: left, top

# Challenge 1

Use debug techniques to find (and possibly solve) the bug in function `evenOdd()`.

---
background-image: url(/assets/images/background_challenge_2.png)
class: left, top

# Challenge 2

Use debug techniques to find (and possibly solve) the bugs in functions `step()` and `steps()`.

---
class: left, top

# Intermezzo: defensive programming

Debugging starts already while writing code! Defensive programming is the best way to solve bunch of (future) bugs: the future-you will praise you.

1. Add pre-conditions to your functions to avoid unexpected and unpredictable errors in later stages (**fail fast principle**: stop the current operation as soon as any unexpected error occurs)
```r
# PSEUDOCODE !!
if (date < today) stop("Date is too early");
if (class input != numeric vector) stop("Invalid type of input: only numeric vectors allowed.")
```

2. Assert statements to validate output at intermediate steps, e.g. output of 3rd party libraries that you use in your project.

---
class: left, top

# Intermezzo: defensive programming - example

Apply defensive programming to the `custom_min()` function here below to avoid unexpected  behaviors:

```
# customized `min()` function for NUMERIC(!) vectors
# to avoid warning while calculating min() of empty vectors
custom_min <- function(x) {
 if (length(x)>0) min(x) else NA
}
> custom_min(c("e", "r", "a")) # passing chars should return an error
[1] "a"
```

---
class: left, top

# Intermezzo: defensive programming - example

The package [assertthat](https://rdocumentation.org/packages/assertthat/versions/0.2.1) is very useful while applying defensive programming in your coding life. What we do is adding assertion(s) to our function.

```
library(assertthat)
# customized `min()` function for NUMERIC(!) vectors
# to avoid warning while calculating min() of empty vectors
custom_min <- function(x) {
 # check x argument (assertion)
 assertthat::assert_that(
 is.numeric(x),
 msg = "Invalid type of input: only numeric vectors allowed."
 )
 # function core follows
 if (length(x)>0) min(x) else NA
}

# only numeric input are allowed, error arises otherwise
custom_min(c("a", "b"))
Error: Invalid type of input: only numeric vectors allowed.
```

---
background-image: url(/assets/images/background_challenge_3.png)
class: left, top

# Challenge 3 - Debug & defensive programming

Let's move to something more familiar: eels tagging data.

The function `get_migrations()` works nicely in example 1, but it fails in other examples. By giving a look to the function, the values passed to the arguments and the error messages, try to understand what is going wrong and add a series of checks (assertions) to fail "fast" and "nicely".

---
class: left, top

# Bonus challenge

Apply defensive programming to avoid unexpected behavior or cryptic error messages to functions described in challenge 1 and 2.

Function crashes with:

1.  "very large" numbers
![](/assets/images/20230525/20230525_challenge3_unexpected_error_big_numbers.png)
2. decimal numbers
![](/assets/images/20230525/20230525_challenge3_unexpected_error_decimal_number.png)
3. negative numbers
4. vectors or list of numbers
![](/assets/images/20230525/20230525_challenge3_unexpected_error_multiple_numbers.png)

---
class: left, top

# Miscellaneous tips: smaller the better

1. Build up code in small pieces, testing along the way, step by step
1. Make big changes in small steps
1. Smaller functions are better: easier to debug, easier to understand

---
class: left, top

# Miscellaneous tips: do not code so _hard_

1. Do not __hard__code numbers: use variables instead (number of iterations, parameter values in simulations, ...). Easier to read and reduce bugs when you use the same number multiple times
1. Avoid using **global variables** in your function
```r
txt <- "INBO"
my_function <- function() {
 paste(txt, "is fantastic!")
}
> my_function()
[1] "INBO is fantastic!"
```
1. Avoid reusing **global variables** names in your function: very confusing and prone to bugs.
```r
txt <- "INBO"
my_function <- function() {
 txt <- "fantastic"
 paste("coding together is", txt)
}
> my_function()
[1] "coding together is fantastic"
> txt
[1] "INBO"
```

---
class: left, top

# Miscellaneous tips: optimize your code

We are typically lazy, aren't we?

A: "Does my code run?"

B: "Yes!"

A: "But it takes hours and GB of memory!"

B: "I don't mind, it runs after all ..."

However, this kind of code could easily crash when running on bigger datasets.

Let's try the [profiler](https://support.posit.co/hc/en-us/articles/218221837-Profiling-with-RStudio#using-the-profiler) built in RStudio!

---
class: left, top

# The package of the month. Hans' choice

Hans came out again with a package I have never heard: [waldo](https://waldo.r-lib.org/). It's a package from the tidyverse collection, which shows you the differences between two objects in a very readable way.

Try this:

```
library(waldo) # install first if needed
waldo::compare(c("a", "b", "c"), c("a", "b"))
```

Another important case is strings vs factors:

![:scale 100%](/assets/images/20230525/20230525_waldo_example.png)

---
class: left, top

# Resources

- Commented [challenges solutions](https://github.com/inbo/coding-club/blob/main/src/20230525/20230525_challenges_solutions.R) available on GitHub
- Edited video recording available on [vimeo](https://vimeo.com/831857629)
- [webinar](https://youtu.be/-yy_3htRHdU) and related [tutorial](https://htmlpreview.github.io/?https://github.com/berkeley-scf/tutorial-R-debugging/blob/master/R-debugging.html) from Chris Paciorek, Department of Statistics, UC Berkley
- [Debugging in RStudio tutorial](https://support.rstudio.com/hc/en-us/articles/205612627-Debugging-with-RStudio) written by posit (company behind RStudio) itself
- [Debugging chapter](https://adv-r.hadley.nz/debugging.html) from Hadley Wickham's book Advanced R, 2nd edition
- Nature's [article](https://www.nature.com/articles/d41586-023-01833-0) with useful tips for better coding with ChatGPT
- [profiler](https://support.posit.co/hc/en-us/articles/218221837-Profiling-with-RStudio#using-the-profiler) tutorial
- [waldo](https://waldo.r-lib.org/) package homepage

---
class: left, top

# Coding club topics 2023: you vote!

No, you do not need to vote this time. You showed a lot of interest for the subject __spatial vector data with sf__, even if it lost twice.

![:scale 70%](/assets/images/20230525/20230525_sf.jpg)

---
class: center, middle

![:scale 30%](/assets/images/coding_club_logo_1.png)

Room: 01.20 - Willy Van Der Meeren (hopefully we'll find something better) 
Date: __27/06/2023__, van 10:00 tot 12:30 
Subject: spatial data with sf 
(registration announced via DG_useR@inbo.be)