R/Exams in Sakai

.blade1.content[
 <div class = 'logosvc'><img src="imgs/logoSVC2019.svg" width="150px" /></div>
]
.blade2[.content.vertical-rl[

]]
.blade3[.universidad-de-murcia[
University of Murcia
 <div class = 'escudoum'><img src="imgs/escudo-um-peq.png" width="265px" /></div>
]]
.blade4[.content.sideways-rl[.center[

]]]
.hole[.content.center.vmiddle[
# Sakai Virtual Conference 2019
## R/Exams in Sakai
<hr />
6th of November 2019
.ponents[
Alacid Cárceles, Valentina

Arnaldos García, Fuensanta

Caballero Pintado, María Victoria

Díaz Delfa, María Teresa

Faura Martínez, Úrsula

Méndez Pérez, Jesús María

Molera Peris, Lourdes

Palazón Ferrando, José Antonio

Parra Frutos, Isabel

Ubero Pascal, Nicolás
]]]

---
class: inverse
# Introduction

---

- Personalized assignments/assessments 

- External questions pool(s)
- Platform/LMS independence 

- Solution: [Exams package (Automatic Generation of Exams in R)](https://cran.r-project.org/web/packages/exams/)

---

.pull-left[ 
- R Package
- Open-source
- Based on (potentially) dynamic exercise templates 
- Output for different learning management systems (LMS)
- [Rexams Web page](http://www.r-exams.org/)   
]

.pull-right[ 
 <div class="figure">
 <img src="imgs/oneforall.svg" alt="One for all?" width="60%" />
 One for all?
 </div>
]

---

<div class="figure" style="text-align: center">
<img src="imgs/oneforallwithSakai.svg" alt="One for all with Sakai!" width="60%" />
One for all with Sakai!
</div>

---

background-image: url("imgs/estadisticas1.svg") 
background-position: 100% 85%
class: shuriken
## Usage Statistics for Tests & Quizzes

### University of Murcia (UM)

- Degrees: 54
- Master degrees: 70
- Instructors: 2642
- Students: 30.164
- [More information](https://www.um.es/web/umu-en-cifras/)

- From October 2018 to October 2019
- Published item questions on Tests & Quizzes tool

---

---

- Why do we want to use R/Exams in Sakai?

.pull-left[ 
### Assignments
- Personalized assignments ("pre-labs" or "follow-up")
- Def: Pre-lab exercises must be completed before each laboratory session so that students will come to the lab prepared.
]

.pull-right[ 
### Tests & Quizzes
- Assessments or question pools generated from dynamic exercise templates
]

---

---

- Every question is written in a file

- Written in two formats

- Markdown `$+$` .blue[R code] `$\rightarrow$` **.Rmd**

- `$\LaTeX$` `$+$` .blue[R code] `$\rightarrow$` **.Rnw**

- .blue[R code] is optional
]

--
 

.pull-right[

<img src="imgs/elearning-create-sakai.svg" width="100%" />
]

---

Question 
     ======== 
     What is the seat of the federal authorities in  
     Switzerland (i.e., the de facto capital)? 
      
     Answerlist 
     ---------- 
     * Basel 
     * Bern 
     * Geneva 
     * Lausanne 
     * Zurich 
     * St. Gallen 
     * Vaduz 
      
     Solution 
     ======== 
     There is no de jure capital but the de facto capital and  
     seat of the federal authorities is Bern. 
      
     Answerlist 
     ---------- 
     * False 
     * True 
     * False 
     * False 
     * False 
     * False 
     * False 
      
     Meta-information 
     ================ 
     exname: Swiss Capital 
     extype: schoice 
     exsolution: 0100000 
     exshuffle: 5 
]

\begin{question} 
     What is the seat of the federal authorities in  
     Switzerland (i.e., the de facto capital)? 
      
     \begin{answerlist} 
       \item Basel 
       \item Bern 
       \item Geneva 
       \item Lausanne 
       \item Zurich 
       \item St.~Gallen 
       \item Vaduz 
     \end{answerlist} 
     \end{question} 
      
     \begin{solution} 
     There is no de jure capital but the de facto capital and  
     seat of the federal authorities is Bern. 
      
     \begin{answerlist} 
       \item False. 
       \item True. 
       \item False. 
       \item False. 
       \item False. 
       \item False. 
       \item False. 
     \end{answerlist} 
     \end{solution} 
      
     \exname{Swiss Capital} 
     \extype{schoice} 
     \exsolution{0100000} 
     \exshuffle{5} 
      
]

---

```{r
questions <- c("Example1.Rmd", "Example2.Rnw", "Example3.Rmd", "Example4.Rmd")
exams2sakai( questions, ... )
exams2nops( questions, ... )
```

- `exams2sakai` `$\rightarrow$`  Our new function
- `exams2nops` `$\rightarrow$`  Written exams with multiple choice (and/or single choice) exercises that can be automatically generated, scanned, and evaluated

- `exams2html` `$\rightarrow$`  Previews and print views
- `exams2qti12` `$\rightarrow$`  Not compatible with Sakai
- `exams2blackboard`
- `exams2moodle`
- ...

---

We are going to create an assessment with four examples

### Steps (test yourself)

1. [Install R and Rstudio](https://rstudio.com/products/rstudio/download/#download) 
1. Download
  - [Rexams package with our modifications](others/exams_4.0.0.tar.gz)
  - [R project zip file with examples](others/svc2019.zip)
]

.pull-right[ 
#### 2. Generating
  
1. Extract R project zip and open R project file
1. Install `Rexams package`
1. Open `2sakai-assessments.R`
1. Run it
1. In your proyect directory a new `qtisakai.zip` file has been generated
]

#### 3. Going to Sakai

1. Extract xml file from `qtisakai.zip` file
1. Import to Tests & Quizzes tool on Sakai (20, 19 or 12 version)

---

### Creating zip file

Files and directories

.column.nops[.content[

#### At the beginning

├── 2sakai-assessments.R
    └─── exercises
        ├── examsExtra.R 
        ├── example1.Rmd
        ├── example2.Rnw
        ├── example3.Rmd
        ├── example4.Rmd
        └── example4
            ├── image.png
            └── points.dat
    
]]

.column.nops[.content.center[

```{r
library( exams )

myexam <- list( "example1.Rmd",
 "example2.Rnw",
 "example3.Rmd",
 "example4.Rmd" )
myexam <- paste( "exercises", myexam, sep = "/" )

set.seed( 101 )

ex1 <- exams2sakai( myexam, n = 1, 
 converter="pandoc-mathjax",
 dir = ".", points = c(1, 1, 1, 1 ), 
 showpoints = TRUE )

ex2 <- exams2html( myexam ) # for a quick check
```
 
]]

.column.nops[.content[

#### After running `2sakai-assessments.R`

```r    
     ├── 2sakai-assessments.R
     ├── exercises
     │    ├── examsExtra.R
     │    ├── example1.Rmd
     │    ├── example2.Rnw
     │    ├── example3.Rmd
     │    ├── example4.Rmd
     │    └── example4
     │        ├── image.png
     │        └── points.dat
*    └── qtisakai.zip
```    
]]

---

.center[ 
<video width="1024" height="500" controls>
 <source src="videos/SVC2019Test&Quizzes.webm" type="video/mp4">
</video>
]

---

### Creating nops files

Files and directories

.column.nops[.content[

#### At the beginning

├── 2nops-assignments.R
    └─── exercises
        ├── examsExtra.R    
        ├── example1.Rmd
        ├── example2.Rnw
        ├── example3.Rmd
        ├── example4.Rmd
        └── example4
            ├── image.png
            └── points.dat
    
]]

.column.nops[.content.center[

```r
library( exams )

myexam <- list( "example1.Rmd", 
 "example2.Rnw",
 "example3.Rmd",
 "example4.Rmd" )
myexam <- paste( "exercises", myexam, sep = "/" )

set.seed( 101 )
ex1 <- exams2nops( myexam, n = 3, 
 dir = "nops", points = c(1, 1, 1, 1 ), 
 showpoints = TRUE )

ex2 <- exams2html( myexam ) # for a quick check
```
 
]]

.column.nops[.content[

#### After running `2nops-assignments.R`

```r    
    ├── 2nops-assignments.R
    ├── exercises
    │   ├── examsExtra.R  
    │   ├── example1.Rmd
    │   ├── example2.Rnw
    │   ├── example3.Rmd
    │   ├── example4.Rmd
    │   └── example4
    │        ├── image.png
    │        └── points.dat
*   └── nops
*       ├── metainfo.rds
*       ├── nops1.pdf
*       ├── nops2.pdf
*       └── nops3.pdf
```
    
]]

---

---

.center[ 
<video width="1024" height="500" controls>
 <source src="videos/SVC2019Assignments.webm" type="video/mp4">
</video>
]

---

1. **Local**. Preparing assignment in nops format, one for each student.

1. **Sakai**. Download student submission from *assignments*

---

3. **Local**. Unzip file to obtain student data

```
    Example
    ├── Student, 0015 (student0015)
    │   └── Submission attachment(s)
    ├── Student, 0016 (student0016)
    │   └── Submission attachment(s)
    └──  Student, 0017 (student0017)
        └── Submission attachment(s)
    ```

1. **Local**. Copy a *nop* file for every student *"Submission attachment(s)"* directory  
   For example, using a `shell-script` en GNU/Linux or Mac, or `PowerShell` in windows

```
    Example
    ├── Student, 0015 (student0015)
    │   └── Submission attachment(s)
    │       └── nops1.pdf
    ├── Student, 0016 (student0016)
    │   └── Submission attachment(s)
    │       └── nops2.pdf
    └── Student, 0017 (student0017)
        └── Submission attachment(s)
            └── nops3.pdf
    ```

1. **Local**. Create a zip file from assignment directory

---

1. **Sakai**. Upload zip file to the *assignment*

---

1. **Sakai**. 0015 Student view example after uploading zip file with personalized assignment.

---

---

Basic single choice example with only plain text

---

Multiple choice example using latex

<<echo=FALSE, results=hide>>= 
 
 functions <- c("$f(x)= \\sqrt{x}$", "$f(x)=x$", "$f(x)=x^2$", "$f(x)= \\ln(x)$") 
 
 derivatives <- c("$f'(x)=1/{2\\sqrt{x}}$", "$f'(x)=1$", "$f'(x)=2x$", "$f'(x)=1/x$") 
 
 wrongDerivatives <- c("$f'(x)=1/{\\sqrt{x}}$", "$f'(x)=x$", "$f'(x)=x^3 `$", "$`f'(x)=e^{x}$") 
 
 ex<-c("Deriverative functions") 
 @ 
 
 \begin{question} 
 Select derivative of the function 
 
 <<echo=FALSE, results=tex>>= 
 ind.correct <- sample(1:length(functions), 1) 
 writeLines(functions[ind.correct]) 
 @ 
 
 <<echo=FALSE, results=tex>>= 
 indices <- 1:length(functions) 
 answers <- c(derivatives[ind.correct], derivatives[-ind.correct], wrongDerivatives) 
 
 booleanAnswers <- c(TRUE, rep(FALSE, length(derivatives) - 1 + length(wrongDerivatives))) 
 
 answerlist(answers, markup = "latex") 
 @ 
 
 \end{question} 
 
 \begin{solution} 
 <<echo=FALSE, results=tex>>= 
 answerlist(ifelse(booleanAnswers, "True", "False" ), markup = "latex", ex) 
 @ 
 \end{solution} 
 
 %%\extype{schoice} 
 %%\exsolution{\Sexpr{mchoice2string(solutions)}} 
 %%\exname{prueba} 
 %%\exshuffle{3} 
 %%\SweaveUTF8 
.blade3[.universidad-de-murcia[
University of Murcia
]]

---

Matrix question example

```{r data generation, echo = FALSE, results = "hide"} 
 ## DATA GENERATION 
 quiz <- c( "Option number 1.", "FALSE", "Feedback number 1.", 
 "Option number 2.", "FALSE", "Feedback number 2.", 
 "Option number 3.", "TRUE", "Feedback number 3.", 
 "Option number 4.", "FALSE", "Feedback number 4.", 
 "Option number 5.", "FALSE", "Feedback number 5." ) 
 
 quiz <- matrix( quiz, ncol = 3, byrow = TRUE ) 
 questions <- quiz[ , 1 ] 
 solutions <- as.logical( quiz[ , 2 ] ) 
 feedback <- quiz[ , 3 ] 
 randomQuiz <- sample( 1:5 ) 
 questions <- questions[ randomQuiz ] 
 solutions <- solutions[ randomQuiz ] 
 feedback <- feedback[ randomQuiz ] 
 ``` 
 
 Question 
 ======== 
 Choose the answer with the number three. 
 
 ```{r questionlist, echo = FALSE, results = "asis"} 
 answerlist( questions, markup = "markdown" ) 
 ``` 
 
 Solution 
 ======== 
 It's very easy: one correct and talks about number three. 
 
 ```{r solutionlist, echo = FALSE, results = "asis"} 
 answerlist( ifelse( solutions, "True", "False" ), markup = "markdown") 
 ``` 
 
 Meta-information 
 ================ 
 extype: schoice 
 exsolution: `r mchoice2string(solutions, single = TRUE)` 
 exname: Matrix example 
.blade3[.universidad-de-murcia[
University of Murcia
]]

---

Calculated question example with hot-spot (not hot-spot question type)

```{r, echo = FALSE, results = "hide", message=FALSE} 
 
 require( imager ) 
 
 exams::include_supplement( "examsExtra.R", recursive = TRUE ) 
 source( "examsExtra.R" ) 
 
 datOut <- hotSpot( "example4/Dragonfly_anatomy-color-mudo.png", 
 "example4/Dragonfly_anatomy.dat" ) 
 ``` 
 
 Question 
 ======== 
 Indicates the correct codes for the structures shown in the figure. 
 
 ![](image.png){ width="80%"} 
 
 ```{r questionlist, echo = FALSE, results = "asis"} 
 answerlist( datOut$questions, markup = "markdown" ) 
 ``` 
 
 Solution 
 ======== 
 Review the topic material (see here). 
 
 ```{r solutionlist, echo = FALSE, results = "asis"} 
 # answerlist( ifelse(dat$solutions, "True", "False"), explanations, markup = "markdown") 
 ``` 
 
 Meta-information 
 ================ 
 extype: mchoice 
 exsolution: `r mchoice2string( datOut$solutions )` 
 exname: image description 
]

labx  laby   finix finiy 
     cabeza             123    80      71    60   
     tórax              271    80     191    60   
     abdomen            692    80     360    60   
     'ojo compuesto'    120   331     157   423 
     antena              80   528     116   430 
     'piezas bucales'   104   583     153   513  
     notos              640   315     639   439  
     ala                758   242     554   261  
     esternos           645   602     505   474 
     patas              340   639     227   579 
     pleuras            331   587     285   458 
     espiráculos        737   318     686   455 
      
]

---

hotSpot <- function( img, dat, nOptions = 5, nTrue = 3, 
 outImage = "image.png", wrong = NULL ){ 
 exams::include_supplement( img, recursive = TRUE ) 
 exams::include_supplement( dat, recursive = TRUE ) 
 
 img <- load.image( img ) 
 dat <- read.table( dat ) 
 
 nFalse <- nOptions - nTrue 
 corrects <- 1:nTrue 
 noCorrects <- ( nTrue + 1 ):nOptions 
 
 datC <- dat[ sample( 1:nrow( dat ), nOptions ), ] 
 datC$solutions <- c( rep( TRUE, nTrue ), 
 rep( FALSE, nFalse ) ) 
 rownames( datC )[ noCorrects ] <- sample( rownames( datC )[ noCorrects ], 
 nFalse ) 
 datC <- datC[ sample( nOptions ), ] 
 
 ## questions/answer 
 questions <- character( nOptions ) 
 questions <- c( paste( 1:nOptions, ": ", 
 rownames( datC ), sep = "" ) ) 
 
 solutions <- logical( nOptions ) 
 solutions <- datC$solutions 
 
 par( mar= c( 0, 0, 0, 0 ) ) 
 png( outImage, width = 1500, height = 1500 ) 
 # Question image 
 plot( img, axes = FALSE ) 
 arrows( datC$labx, datC$laby, datC$finix, datC$finiy, 
 length = 0.25, angle = 15, col = "white", lwd = 10 ) 
 points( datC$labx, datC$laby, pch = 20, col= "white", 
 cex = 16 ) 
 text( datC$labx, datC$laby, 1:nrow( datC ), pch = 20, 
 col= 1, cex = 4 ) 
 dev.off() 
 
 randomSort <- order( questions ) 
 return( data.frame( questions[ randomSort ], solutions[ randomSort ] ) ) 
 } 
.blade3[.universidad-de-murcia[
University of Murcia
]]

---

Great versatility:

- Advanced calculated questions using any function available in R

- Graphics embedded in the questions (generated with R)

etc.

---

- New template _qtisakai.xml_ has been created

- New function _exams2sakai_ has been created

- Single choice and multiple choice type questions work

- Added new metadata to identify different multiple choice types: _mca_ (multiple correct answer) or _mcss_ (multiple correct single selection)

- Added correct or incorrect feedback (Question-Level Feedback)

---

- Scripts to randomize/assign a nops file to each student

- Create R scripts with functions to generate new schema questions

- More testing!

- Testing [predefined examples](http://www.r-exams.org/templates/) from R/Exams web page

- Identify the different metadata types in R/Exams and study how to use them in Sakai

- Adding Selection-Level (A, B, C...) Feedback and Both Feedback (in progress)

- Importing zip doesn't work

- Study if it is possible to adapt the different question types in Sakai to R/Exams (Numeric Response is in progress)

- Maximum number of answers is defined by upper case letters from A to Z (26 answers)

---

class: shuriken
background-image: url("imgs/uneteCorriendo.jpg") 
background-size: cover
## Join us!

class: shuriken
background-image: url("imgs/niklas-braun-VRya2cVDHbo-unsplash.jpg") 
background-size: cover