---
title: "Import CSV from ConfoMap ISO25178"
author: "Ivan Calandra"
date: "`r Sys.Date()`"
output:
  html_document:
    toc: true
    toc_depth: 5
    toc_float: true
    theme: cerulean
    highlight: pygments
params:
  data: 
    value: "Data/CoordSys_FLT1-7_50x.csv"
---

<!-- Define the CSV-file to use on line 14 --> 
<!-- This CSV-file must be in a 'Data' folder within the directory of the present Rmd file --> 

<!-- Lines 75, 84, 93-99, 106-109 and 116 (at least) might need to be edited with new data --> 

---

CSV-file used for this script: **`r paste0("~/", params$data)`**


```{r Knitr Options, include=FALSE}
	knitr::opts_chunk$set(comment=NA, message=FALSE, indent="", error=TRUE)
```

---


### 1. Goal of the script
This script formats the output of the resulting CSV-file from applying a template computing ISO25178 parameters in ConfoMap.
The script will:

1. Read in the original CSV-file (step #3)  
2. Format the data (step #4)  
3. Write an XLSX-file and save an R object ready for further analysis in R (step #5)

All files will be saved in a subfolder "Data" within the directory of the present Rmd file, i.e. "`r paste0("~/",dirname(params$data))`".


---


### 2. Load packages
```{r}
library(openxlsx)
library(tools)
library(R.utils)
```


---


### 3. Read in original CSV-file
```{r}
imp.info <- file.info(params$data)
imp.data <- read.csv(params$data, header=FALSE, skip=3, stringsAsFactors=FALSE, na.strings="*****")
```

The imported file is: "`r paste0("~/", params$data)`"  
Its modification, 'last status change' (= 'creation' on Windows) and last access times are, respectively: "`r imp.info$mtime`", "`r imp.info$ctime`" and "`r imp.info$atime`".


---


### 4. Format data
#### 4.1. Keep only interesting columns
```{r}
keep.col <- c(1:2, 4:5, 7:9, 11:13, 15:53)
data.keep <- imp.data[keep.col]
str(data.keep)
```


#### 4.2. Add headers
```{r}
head.keep <- strsplit(readLines(params$data, n=3)[2], ",")[[1]][keep.col]
head.keep <- c("Analysis date","Analysis time", head.keep[-(1:2)])
names(data.keep) <- gsub("\\.+", "\\.", make.names(head.keep))
names(data.keep) <- gsub("\\.$", "", names(data.keep))
names(data.keep)
```


#### 4.3. Save units
```{r}
col.var <- 5:length(data.keep)
names.var <- names(data.keep)[col.var]
units.var <- strsplit(readLines(params$data, n=3)[3], ",")[[1]][keep.col]
units.var <- units.var[col.var]
units.var <- c(units.var, "µm", "µm") ## in planing for X/Y shift! ##
units.var.table <- data.frame(variable=c(names(data.keep)[col.var],"Xshift","Yshift"), unit=units.var)
names(units.var) <- c(names.var,"Xshift","Yshift")
units.var.table
```


#### 4.4. Split the column 'Analyzed.file' into several columns
```{r}
split.info <- strsplit(data.keep$Name.of.the.studiable,"_| > ")
data.final <- data.frame(do.call(rbind, split.info), data.keep[-3])
data.final <- data.final[-(6:7)]
names(data.final)[1:5] <- c("Sample","Microscope","Objective","Replica","Acquisition.date")
str(data.final)
```


#### 4.5. Shorten the variable names
```{r}
col.var2 <- 19:47
names(data.final)[col.var2] <- sapply(strsplit(names(data.final)[col.var2], ".", fixed=TRUE), `[[`, 1)
names(data.final)
```


#### 4.6. Format date columns
```{r}
data.final$Acquisition.date <- as.Date(as.character(data.final$Acquisition.date), format="%Y%m%d")
data.final$Analysis.date <- as.Date(data.final$Analysis.date, format="%d/%m/%Y")
data.final$Date.of.the.measurement <- as.POSIXct(data.final$Date.of.the.measurement, format="%d/%m/%Y %H:%M:%S")
str(data.final)
```


#### 4.7. Calculate X/Y shifts
```{r}
data.final[1:10,"Xshift"] <- abs(data.final[1:10,"Distance.A"] - data.final[1,"Distance.A"])
data.final[11:20,"Xshift"] <- abs(data.final[11:20,"Distance.A"] - data.final[11,"Distance.A"])
data.final[1:10,"Yshift"] <- abs(data.final[1:10,"Distance.B"] - data.final[1,"Distance.B"])
data.final[11:20,"Yshift"] <- abs(data.final[11:20,"Distance.B"] - data.final[11,"Distance.B"])
data.final[c("Microscope","Replica","Xshift","Yshift")]
```


#### 4.8. Add units as comment()
```{r}
comment(data.final) <- units.var
```

Type `comment(data.final)` to check the units of the columns.
```{r, echo=FALSE}
comment(data.final)
```


#### 4.9. Check the result
```{r}
str(data.final)
head(data.final)
```


---


### 5. Save data
#### 5.1. Format file output name
```{r}
file.out <- file_path_sans_ext(row.names(imp.info))
```
The files will be saved as "`r paste0("~/",file.out,".[ext]")`".


#### 5.2. Write to XLSX
```{r}
write.xlsx(list(data=data.final, units=units.var.table), file=paste0(file.out,".xlsx"))
xlsx.info <- file.info(paste0(file.out,".xlsx"))
```

The exported XLSX-file is: "`r basename(row.names(xlsx.info))`"  
Its modification, 'last status change' (= 'creation' on Windows) and last access times are, respectively: "`r xlsx.info$mtime`", "`r xlsx.info$ctime`" and "`r xlsx.info$atime`".


#### 5.3. Save R object
```{r}
saveObject(data.final, file=paste0(file.out,".Rbin"))
Rbin.info <- file.info(paste0(file.out,".Rbin"))
```

The exported Rbin-file is: "`r basename(row.names(Rbin.info))`"  
Its modification, 'last status change' (= 'creation' on Windows) and last access times are, respectively: "`r Rbin.info$mtime`", "`r Rbin.info$ctime`" and "`r Rbin.info$atime`".


---

```{r}
sessionInfo()
```

---

END OF SCRIPT