class: center, middle  <!-- Do not forget to adapt the presentation title in the header! --> <!-- Adjust the presentation to the session. Focus on the challenges, this is not a coding tutorial. Note, to include figures, store the image in the `/docs/assets/images/yyyymmdd/` folder and use the jekyll base.url reference as done in this template or see https://jekyllrb.com/docs/liquid/tags/#links. using the scale attribute , you can adjust the image size. --> <!-- Adjust the day, month --> # 14 DECEMBER 2023 ## INBO coding club <!-- Adjust the room number and name --> Herman Teirlinck Building 01.05 - Isala Van Diest --- class: left, top ## ROOMIE: room reservation ``` > if (isFALSE(roomie)) { + warning("Please confirm asap the room reservation on the roomie") + } Warning message: Please confirm asap the room reservation on the roomie ``` --- class: center, middle # Do you know that?  Go to the [Google document](https://docs.google.com/document/d/131PsEgMVHw-aMXEZxQ7ea_wdrnwVD1j6NFhiR7Cn5J0/edit)! --- class: center, middle <!-- Create a new badge using Inkscape or other programs based on the assets/images/coding_club_badges.svg file -->  --- class: left, top ### How to get started? Check the [Each session setup](https://inbo.github.io/coding-club/gettingstarted.html#each-session-setup) to get started. ### First time coding club? Check the [First time setup](https://inbo.github.io/coding-club/gettingstarted.html#first-time-setup) section to setup. --- class: left, top  --- class: center, top # Share your code during the coding session <!-- Create a new hackmd file and replace this link (twice!) --> Go to https://hackmd.io/nDpNbgX7T8m0ozuvKOJClA?edit and start by adding your name in section "Participants". <iframe src="https://hackmd.io/nDpNbgX7T8m0ozuvKOJClA?edit" height="400px" width="800px"></iframe> --- class: left, top # QGIS - [QGIS](https://qgis.org/en/site/) is a free and open source geographic information system. - Typically described as the open source version of [ArcGIS](https://www.arcgis.com/index.html). - QGIS allows to create, edit, visualise, analyse and publish geospatial information. - The most recent version at the moment of writing (2023-12-7) is 3.34 --- class: left, top # What is qgisprocess? - [qgisprocess](https://r-spatial.github.io/qgisprocess/index.html) is an R package - it wraps the standalone [qgis_process command-line utility](https://docs.qgis.org/3.28/en/docs/user_manual/processing/standalone.html) - it allows you to run processing algorithms and models (built-in or provided by plugins) directly from R without starting the qgis_process command-line utility or the QGIS Desktop itself --- class: left, top # Why should I use it? - Expand geospatial processing abilities in R - Still have all other processing capabilities of R - Reproducibility - No QGIS project needed - Unified interface to QGIS natice algorithms and external processing providers/plugins, e.g. GRASS GIS, SAGA, GDAL <small> Source: [Floris' presentation](https://florisvdh.github.io/foss4g-2023-qgisprocess/#17) at FOSS4G 2023</small> --- class: center, top  Download cheatsheet in [English](https://github.com/inbo/coding-club/blob/master/cheat_sheets/20231214_cheat_sheet_qgisprocess_en.pdf) or in [Spanish](https://github.com/inbo/coding-club/blob/master/cheat_sheets/20231214__cheat_sheet_qgisprocess_es.pdf) --- class: left, top # Download data and code You can download the material of today: - automatically via `inborutils::setup_codingclub_session()`<small>\*</small> - manually<small> \*\*</small> from GitHub folders [coding-club/data/20231214](https://github.com/inbo/coding-club/tree/master/data/20231214) and [coding-club/src/20231214](https://github.com/inbo/coding-club/tree/master/src/20231214) <br> <small> \* You can use the date in "YYYYMMDD" format to download the coding club material of a specific day, e.g. run `setup_codingclub_session("20220428")` to download the coding club material of April, 28 2022. If date is omitted, i.e. `setup_codingclub_session()`, the date of today is used. For all options, check the [tutorial online](https://inbo.github.io/tutorials/tutorials/r_setup_codingclub_session/).</small> <br> <small> \*\* Check the getting started instructions on [how to download a single file](https://inbo.github.io/coding-club/gettingstarted.html#each-session-setup)</small> --- class: left, top # Data and scripts description - `water`: [20231214_watersurfaces_hab.gpkg](https://github.com/inbo/coding-club/tree/master/data/20231214/20231214_watersurfaces_hab.gpkg). Standing water Natura 2000 habitat types (2190_a and 31xx) and regionally important biotopes (rbbah) in Flanders<small> \*</small> - `hab_springs`: [20231214_habitatsprings_flanders.geojson](https://github.com/inbo/coding-club/tree/master/data/20231214/20231214_habitatsprings_flanders.geojson). A geospatial collection of points that corresponds with the presence of the Natura 2000 habitat type 7220 (Petrifying springs with tufa formation (Cratoneurion)) in springs and streaming water segments in the Flanders<small> \*\*</small> - `ludwigia_grandiflora`: [20231214_ludwigia_grandiflora.geojson](https://github.com/inbo/coding-club/blob/master/data/20231214/20231214_Ludwigia_grandiflora.geojson). Dataset with GBIF observations of water primrose (EN) - waterteunisbloem (NL) (_Ludwigia grandiflora_) in Flanders from 2021 to October 2022<small> \*\*\*</small> <small> \* Westra, T., Vanderhaeghe, F., & Herr, C. (2021). Map of standing water habitat types and regionally important biotopes in Flanders (watersurfaces_hab_v4) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4915849.</small> <br> <small> \*\* Derived from "Oosterlynck, P. (2020). Distribution of the Natura 2000 habitat type 7220 (Cratoneurion) in Flanders and Brussels Capital Region, Belgium (version 2020) (habitatsprings_2020v2) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.3784149".</small> <br> <small> \*\*\* Source: GBIF.org (24 October 2022) GBIF Occurrence Download https://doi.org/10.15468/dl.sg4x9v. </small> --- class: left, top # Data and scripts description - `land_use`: [20231214_land_use_nara_2016_100m.tif](https://github.com/inbo/coding-club/blob/main/data/20231214/20231214_land_use_nara_2016_100m.tif). A categorical raster of land use of Flanders with 9 land use categories. Thanks, NARA team! - `prot_areas`: [20231214_natura2000_protected_areas.gpkg](https://github.com/inbo/coding-club/blob/main/data/20231214/20231214_natura2000_protected_areas.gpkg). A geopackage with NATURA2000 protected areas - `temperature`: [20231214_tdiff_tmax_tmin_january.tif](https://github.com/inbo/coding-club/blob/main/data/20231214/20231214_tdiff_tmax_tmin_january.tif). A raster with January average maximum and minimum temperature at 10 minutes precision* - [20231214_challenges.R](https://github.com/inbo/coding-club/blob/master/src/20231214/20231214_challenges.R): a R script to start with. <small> \* Source: [WorldClim](https://worldclim.org/data/worldclim21.html) historical data.</small> --- class: left, top # Load packages Before installing and loading qgisprocess, we load some other packages (if needed, install them) we will use today: ```r library(tidyverse) library(sf) # to deal with spatial vector data library(terra) # to deal with raster data ``` Optional: for visualization purposes, you could use the `mapview()` function from [mapview](https://r-spatial.github.io/mapview/index.html) R package. Install it, if needed, and load it: ```r library(mapview) ``` --- class: left, top # Challenge 0 - Install the package: install.packages("qgisprocess") - Load the package: ```r > library(qgisprocess) Attempting to load the package cache ... Success! QGIS version: 3.30.3-'s-Hertogenbosch Having access to 349 algorithms from 4 QGIS processing providers. Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details. Warning message: package ‘qgisprocess’ was built under R version 4.3.1 ``` - Algorithms come from processing providers. What are they? ```r > qgis_providers() # A tibble: 4 × 3 provider provider_title algorithm_count <chr> <chr> <int> 1 gdal GDAL 56 2 qgis QGIS 50 3 3d QGIS (3D) 1 4 native QGIS (native c++) 242 ``` --- class: left, top # Challenge 0 Run the buffer algorithm on the New York City shapefile. Example from qgisprocess homepage. ```r input <- sf::read_sf(system.file("shape/nc.shp", package = "sf")) result <- qgis_run_algorithm( "native:buffer", INPUT = input, DISTANCE = 1, DISSOLVE = TRUE ) result #> <Result of `qgis_run_algorithm("native:buffer", ...)`> #> List of 1 #> $ OUTPUT: 'qgis_outputVector' chr "/tmp/RtmpX60EOv/file2d652e33f3c4/file2d6572d31d21.gpkg" output_sf <- sf::st_as_sf(result) plot(sf::st_geometry(output_sf)) ``` --- background-image: url(/assets/images/background_challenge_1.png) class: left, top # Challenge 1 1. In challenge 0 we specified (a little blindly) some arguments, e.g. `DISSOLVE = TRUE`. What does this argument do? What happens if you set `DISSOLVE = FALSE`? How can you get a description of the chosen algorithm? How can you retrieve information about all the arguments allowed by the algorithm? Or about the generated output? Hint: use the cheat sheet! 2. Calculate the ‘shortest’ path (lines) between the `hab_springs` points and the nearest watersurface `water`. 3. Calculate the ‘shortest’ paths (lines) between the polygons `water` and the `hab_springs` points. 4. We are all pipe (`%>%`)* lovers, aren't we? Can you use `qgis_run_algorithm()` in a pipe? No. How to combine qgisprocess and pipes? Hint: check the cheat sheet. <small> \* Maybe you have already seen the new R pipe: `|>`. This pipe is available with R 4.0 or higher. </small> --- class: left, top # INTERMEZZO 1 - qgisprocess vs sf/terra The sf package is widely used in geospatial R community. We use it often, see e.g. the INBO coding club ["Spatial data in R"](https://inbo.github.io/coding-club/sessions/20230627_spatial_data_in_r.html#1), 2023-06-27. The same goes for the terra package, the mainstream package to work with rasters. See e.g. the INBO coding club ["rasters in R"](https://inbo.github.io/coding-club/sessions/20221124_coding_club_rasters.html#1), 2022-11-24. When should I use qgisprocess instead of native sf/terra functions? - Not all QGIS algorithms have a sf/terra counterpart - Work with big spatial files - QGIS algorithms are typically faster - You are a QGIS pro and don't want to/have time to learn sf/terra or other packages --- class: left, top # INTERMEZZO 1 - qgisprocess vs sf/terra Example: calculate buffers at high resolution ``` > tic("buffer via sf") > buffer_via_sf <- sf::st_buffer(water, nQuadSegs = 5000, dist = 100) > toc() buffer via sf: 49.92 sec elapsed > tic("buffer via qgisprocess") > buffer_via_qgisprocess <- qgis_run_algorithm( + INPUT = water, + algorithm = "native:buffer", + SEGMENTS = 5000, + DISTANCE = 100 + ) > toc() buffer via qgisprocess: 28.31 sec elapsed ``` On the other hand, sf is faster for buffers at low resolution. --- background-image: url(/assets/images/background_challenge_2.png) class: left, top # Challenge 2 1. **Count** the number of occurrences of Ludwigia grandiflora in each of the water polygons. Hint: use `qgis_search_algorithms` to find the right algorithm to use. 2. **Count** the number of individuals (column `individualCount`) of Ludwigia grandiflora in each of the water polygons. Hint: use the same algorithm used for previous exercise. 3. Create a buffer around hab_springs points with a dynamic radius expressed as the square root of `area_m2`/3.14 (constant pi seems not interpreted by QGIS). Hint: check https://r-spatial.github.io/qgisprocess/articles/qgis_expressions.html#example-applying-a-data-defined-override --- class: left, top # INTERMEZZO 2 - the qgis package Using functions with explicit arguments is really user friendly. You will then probably love the [qgis](https://jancaha.github.io/r_package_qgis/index.html) package. It aims to provide calls and simple documentation for each QGIS function available in the QGIS core and the most important algorithm providers like GRASS GIS and SAGA. It requires the qgisprocess package. An example. Instead of: ```r sf::read_sf(system.file("shape/nc.shp", package = "sf")) %>% qgisprocess::qgis_run_algorithm_p( algorithm = "native:buffer", DISTANCE = 0.2, DISSOLVE = TRUE, END_CAP_STYLE = "Flat") %>% sf::st_as_sf() %>% plot() ``` ```r sf::read_sf(system.file("shape/nc.shp", package = "sf")) %>% qgis::qgis_buffer( DISTANCE = 0.2, DISSOLVE = TRUE, END_CAP_STYLE = "Flat") %>% st_as_sf() %>% plot() ``` --- background-image: url(/assets/images/background_challenge_3.png) class: left, top # Challenge 3 Let's work with rasters! What is a raster? "A raster is a spatial (geographic) data structure that divides a region into rectangles called "cells" (or "pixels") that can store one or more values for each of these cells."* 1. How can you get general *statistics* about the temperature difference in January (first layer of 20231214_tdiff_tmax_tmin_january.tif) 2. How can you do the same for the other two layers (minimum/maximum temperature)? 3. Let's do some [zonal statistics](https://up42.com/blog/an-introduction-to-zonal-statistics#what-are-zonal-statistics): calculate the most and the least occurring land use category in each of the Natura2000 protected areas.<small> \*\*</small> <small> \* Source: https://rspatial.org/raster/pkg/1-introduction.html</small> <br> <small> \*\* Notice that if you want to do the same purely in R, you need to use the R package [extractr](https://isciences.gitlab.io/exactextractr/index.html). See the last exercise in [slide 14](https://inbo.github.io/coding-club/sessions/20221124_coding_club_rasters.html#14) (Nov 24, 2022) or the last exercise of [slide 13](https://inbo.github.io/coding-club/sessions/20211125_rasters_in_r.html#13) (Nov 25, 2021).</small> --- class: left, top # Intermezzo 3 - Use QGIS GUI parameters - You may be a pro using QGIS. In that case, you will be happy to know that you can take parameters from the QGIS GUI and use them in qgisprocess! Check Floris' slides [48-50](https://florisvdh.github.io/foss4g-2023-qgisprocess/#48). --- class: left, top # Resources - [Solutions](https://github.com/inbo/coding-club/blob/main/src/20231214/20231214_challenges_solutions.R) are online - [Edited webinar](https://vimeo.com/900825991?share=copy) available on vimeo. Do you know that all INBO coding club videos can be found on our [vimeo channel](https://vimeo.com/user/8605285/folder/1978815)? - [qgisprocess](https://r-spatial.github.io/qgisprocess/index.html) R package documentation - [Interfacing QGIS processing from R](https://florisvdh.github.io/foss4g-2023-qgisprocess/#1): Floris' presentation at FOSS4G 2023. Floris rocks: watch his talk, here is the [video](https://www.youtube.com/watch?v=Qt5DzWThWqI) :-) - [qgis](https://jancaha.github.io/r_package_qgis/index.html) R package documentation - [QGIS](https://qgis.org/en/site/) website - How to get help: use our internal INBO R mailing list (DG_useR) and also check the [FAQs](https://github.com/r-spatial/qgisprocess/issues/new/choose) on the "Issues" page of the qgisprocess GitHub repository --- class: left, top # Holidays! It was a great 2024 with many new people and interesting discussions. Thanks to all of you to make the INBO coding club a lively, welcoming place where we can experiment together, share code and learn from each other. Enjoy the Christmas holidays and see you in 2024!  --- class: center, middle  <!-- Adjust the room and date --> Room: 01.70 - Ferdinand Peeters<br> Date: __30/01/2024__, van 10:00 tot 12:30<br> Subject: **to be decided** <br> (registration announced via DG_useR@inbo.be)