ADT-Analysis

library(ggplot2)
library(dplyr)
library(broom)
library(ggpubr)
library(tidyverse)
library(readr)

data_adt <- read.csv("./ADT-test-results.csv", na.strings = c("N/A"))
data_adt <- data_adt %>% 
  filter(!(is.na(PEOU) | is.na(PU) | is.na(ITU))) %>%
    select(!ID)


SIZE = 18
EQ_SIZE = 6

PEU: perceived ease of use
PU: perceived usefulness
ITU: intention to use
Effectiveness: how high is the score of the subject
Efficiency: how fast is the subject

Boxplots of effectiveness and efficiency variables

Average total effectiveness is above 0.75, indicating a generally good level of comprehensibility of ADT principles. Furthermore, the boxplot shows that effectiveness in context is provides the highest contribution, followed by effectiveness not in context and transferability effectiveness. This suggests that while participants understand ADT principles, they find more difficulties in applying them in practice.

For the efficiency variable, we see a similar situation to that observed for effectiveness, thereby confirming that ADT ``in action’’ is perceived as a more difficult task.

Boxplot of the three variables: PEOU, PU, ITU

The three variables are clearly all above the average value of the Likert scale (i.e., 3), thus suggesting that the general degree of acceptance of the ADT notation is high. We see that, while ITU and PU have comparable values, PEOU receives the highest score. This further confirms that ADTs are considered easy to use by the participants.

Descriptive statistics for all the variables

df <- data_adt
col_names <- c("Stat", colnames(df))
# Create a vector with the desired summary statistics
stats <- c("Min", "1st Qu", "Median", "Mean", "3rd Qu", "Max")

# Create an empty matrix to store the summary statistics
summary_mat <- matrix(NA, nrow = length(stats), ncol = ncol(df))

# Iterate over each column of the original dataframe
for (i in 1:ncol(df)) {
  # Compute the summary statistics
  summary_stats <- summary(df[[i]])
  
  # Store the summary statistics in the matrix
  summary_mat[, i] <- summary_stats
}

# Convert the matrix to a dataframe
summary_df <- as.data.frame(summary_mat)

# Add a row with the mean of each column
#summary_df <- rbind(summary_df, colMeans(df))

# Rename the row names for better readability
row.names(summary_df) <- c(stats)

colnames(summary_df) <- c("PEOU", "PU", "ITU", "Effectiveness", "Efficiency", "Effv. not in Context", "Effc. not in Context", "Transferability Effv.", "Transferability Effc.", "Effv. in Context", "Effc. in Context")

# Transpose the summary dataframe
transposed_df <- t(summary_df)

# Print the transposed dataframe
View(transposed_df)

knitr::kable(transposed_df)

	Min	1st Qu	Median	Mean	3rd Qu	Max
PEOU	2.875	4.000	4.250	4.18000	4.500	5.000
PU	2.929	3.714	4.000	3.91996	4.071	4.571
ITU	3.000	3.625	3.875	3.88000	4.125	5.000
Effectiveness	0.287	0.769	0.792	0.76780	0.833	0.986
Efficiency	0.011	0.101	0.118	0.13116	0.172	0.241
Effv. not in Context	0.625	0.750	0.750	0.78336	0.833	0.958
Effc. not in Context	0.024	0.069	0.094	0.10320	0.150	0.188
Transferability Effv.	0.000	0.556	0.667	0.61348	0.778	1.000
Transferability Effc.	0.000	0.018	0.023	0.02604	0.036	0.049
Effv. in Context	0.111	0.889	0.889	0.90672	1.000	1.000
Effc. in Context	0.009	0.178	0.250	0.26412	0.333	0.500

Test of significance against average value 3 of PEOU, PU and ITU

The normality check, performed with the Kolmogorov-Smirnov test, fails for all the variables with p-value well below the 0.05 significance level. We therefore need to apply a nonparametric test(Wicoxon signed rank) to check whether the three variables are above the average value of the Livert scale (i.e., 3).

ks.test(data_adt$PEOU, "pnorm")

## 
##  Asymptotic one-sample Kolmogorov-Smirnov test
## 
## data:  data_adt$PEOU
## D = 0.99798, p-value < 2.2e-16
## alternative hypothesis: two-sided

ks.test(data_adt$PU, "pnorm")

## 
##  Asymptotic one-sample Kolmogorov-Smirnov test
## 
## data:  data_adt$PU
## D = 0.9983, p-value < 2.2e-16
## alternative hypothesis: two-sided

ks.test(data_adt$ITU, "pnorm")

## 
##  Asymptotic one-sample Kolmogorov-Smirnov test
## 
## data:  data_adt$ITU
## D = 0.99865, p-value < 2.2e-16
## alternative hypothesis: two-sided

Data are not normally distributed, therefore we use a nonparametric test (Wicoxon signed rank).

# Set the desired significance level
alpha <- 0.05

result <- wilcox.test(data_adt$PEOU, mu = 3, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$PEOU
## V = 299, p-value = 1.077e-05
## alternative hypothesis: true location is greater than 3

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that PEOU is lower than the average score is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis that PEOU is lower than the average score is rejected (p < 0.05 )

result <- wilcox.test(data_adt$PU, mu = 3, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$PU
## V = 324, p-value = 7.109e-06
## alternative hypothesis: true location is greater than 3

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that PU is lower than the average score is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis that PU is lower than the average score is rejected (p < 0.05 )

result <- wilcox.test(data_adt$ITU, mu = 3, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$ITU
## V = 300, p-value = 9.282e-06
## alternative hypothesis: true location is greater than 3

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that ITU is lower than the average score is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis that ITU is lower than the average score is rejected (p < 0.05 )

For all the tests we can reject the NULL hypothesis. More specifically, we reject the following hypotheses.

ADT is perceived as difficult to use
ADT is perceived as not useful
There is no intention to use ADT

PEOU and PU against efficiency and effectiveness, using a linear model as a fitting/interpolation function

We now explore the relation with total effectiveness.

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = Effectiveness, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "effectiveness"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
  stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(2.5, 2.75), label.x = c(0.6))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of effectiveness against PEOU

mdl <- lm(PEOU ~ Effectiveness, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ Effectiveness, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.08582 -0.30712  0.02282  0.23528  1.21186 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)     2.8235     0.6206   4.549 0.000143 ***
## Effectiveness   1.7667     0.7968   2.217 0.036772 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5216 on 23 degrees of freedom
## Multiple R-squared:  0.1761, Adjusted R-squared:  0.1403 
## F-statistic: 4.916 on 1 and 23 DF,  p-value: 0.03677

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["Effectiveness", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is rejected (p < 0.05 )

Statistical analysis of effectiveness against PU

mdl <- lm(PU ~ Effectiveness, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ Effectiveness, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.77501 -0.14883  0.03019  0.13340  0.55648 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)     3.1724     0.4211   7.534 1.18e-07 ***
## Effectiveness   0.9736     0.5406   1.801   0.0848 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3539 on 23 degrees of freedom
## Multiple R-squared:  0.1236, Adjusted R-squared:  0.08549 
## F-statistic: 3.244 on 1 and 23 DF,  p-value: 0.08483

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["Effectiveness", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

From the plot, we see that when effectiveness grows also PEOU and PU grows. From the statistical analysis, however, we see that while the relationship between effectiveness and PEOU is significant, the one with PU is not. Therefore the first NULL hypothesis is rejected, while the second NULL hypothesis cannot be rejected.

PEOU will not be determined by effectiveness: reject
PU will not be determined by effectiveness: fail to reject

Relation with efficiency

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = Efficiency, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "efficiency"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
   stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3, 3.25), label.x = c(0.15))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of efficiency against PEOU

mdl <- lm(PEOU ~  Efficiency, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ Efficiency, data = data_adt)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -1.1082 -0.3505  0.1054  0.2439  1.0464 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   3.8272     0.2753  13.901 1.11e-12 ***
## Efficiency    2.6902     1.9231   1.399    0.175    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5517 on 23 degrees of freedom
## Multiple R-squared:  0.07841,    Adjusted R-squared:  0.03834 
## F-statistic: 1.957 on 1 and 23 DF,  p-value: 0.1752

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["Efficiency", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Statistical analysis of efficiency against PU

mdl <- lm(PU ~  Efficiency, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ Efficiency, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.00366 -0.18575  0.07646  0.13986  0.64391 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   3.9532     0.1885  20.972   <2e-16 ***
## Efficiency   -0.2532     1.3167  -0.192    0.849    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3777 on 23 degrees of freedom
## Multiple R-squared:  0.001605,   Adjusted R-squared:  -0.0418 
## F-statistic: 0.03697 on 1 and 23 DF,  p-value: 0.8492

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["Efficiency", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

From the plot, we see that while PEOU grows with efficiency, PU appears to slightly decrease. From the statistical analysis, however, we see that none of the relationships are significant. Therefore none of the NULL hypothesis can be rejected.

PEOU will not be determined by efficiency: fail reject
PU will not be determined by efficiency: fail to reject

Relation between PU and PEOU

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of PU against PEOU

mdl <- lm(PU ~ PEOU, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ PEOU, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.05101 -0.12078  0.06576  0.13676  0.64591 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  3.61384    0.57471   6.288 2.04e-06 ***
## PEOU         0.07323    0.13631   0.537    0.596    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3757 on 23 degrees of freedom
## Multiple R-squared:  0.01239,    Adjusted R-squared:  -0.03054 
## F-statistic: 0.2886 on 1 and 23 DF,  p-value: 0.5962

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["PEOU", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

We see that although there is a slight increase of PU when PEOU grows, the relationship has no statistical significance. Therefore:

PU will not be determined by PEOU: fail to reject

Relation between ITU and PEOU; ITU and PU

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = value, y = ITU, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "indicator"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
  stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3, 3.25), label.x = c(3.5))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of PU against ITU

mdl <- lm(ITU ~ PU, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = ITU ~ PU, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.33404 -0.19893 -0.07393  0.17422  0.55933 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   0.5041     0.5456   0.924    0.365    
## PU            0.8612     0.1386   6.214 2.44e-06 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.2513 on 23 degrees of freedom
## Multiple R-squared:  0.6267, Adjusted R-squared:  0.6105 
## F-statistic: 38.61 on 1 and 23 DF,  p-value: 2.436e-06

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["PU", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is rejected (p < 0.05 )

Statistical analysis of PEOU against ITU

mdl <- lm(ITU ~ PEOU, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = ITU ~ PEOU, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.82323 -0.17595  0.03742  0.19187  1.10351 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   2.8950     0.5940   4.874 6.38e-05 ***
## PEOU          0.2356     0.1409   1.672    0.108    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3883 on 23 degrees of freedom
## Multiple R-squared:  0.1084, Adjusted R-squared:  0.06967 
## F-statistic: 2.797 on 1 and 23 DF,  p-value: 0.108

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["PEOU", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

We clearly see from the plot above that both PEOU and and PU tend to grow with ITU. However, from the analysis below, we see that PEOU and PU do not have any relationship. Therefore:

PU will not be determined by PEOU: fail to reject
ITU will not be determined by PEOU: fail to reject
ITU will not be determined by PU: reject

Relation between PU and PEOU

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of PU against PEOU

mdl <- lm(PU ~ PEOU, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ PEOU, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.05101 -0.12078  0.06576  0.13676  0.64591 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  3.61384    0.57471   6.288 2.04e-06 ***
## PEOU         0.07323    0.13631   0.537    0.596    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3757 on 23 degrees of freedom
## Multiple R-squared:  0.01239,    Adjusted R-squared:  -0.03054 
## F-statistic: 0.2886 on 1 and 23 DF,  p-value: 0.5962

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["PEOU", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Evaluation by single measures of effectiveness/efficiency

PEOU will not be determined by effectiveness not in context: fail to reject
PU will not be determined by effectiveness not in context: fail to reject
PEOU will not be determined by efficiency not in context: fail to reject
PU will not be determined by efficiency not in context: fail to reject
PEOU will not be determined by effectiveness in context: reject
PU will not be determined by effectiveness in context: fail to reject
PEOU will not be determined by efficiency in context: fail to reject
PU will not be determined by efficiency in context: fail to reject
PEOU will not be determined by transferability effectiveness: fail to reject
PU will not be determined by transferability effectiveness: reject
PEOU will not be determined by transferability efficiency: fail to reject
PU will not be determined by transferability efficiency: fail to reject

Relation with effectiveness not in context

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = effectiveness_not_context, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "effectiveness not in context"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
  stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3, 3.25), label.x = c(0.6))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of effectiveness not in context against PEOU

mdl <- lm(PEOU ~ effectiveness_not_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ effectiveness_not_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.26485 -0.24938  0.05538  0.26902  0.89651 
## 
## Coefficients:
##                           Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                 4.5232     1.1056   4.091 0.000448 ***
## effectiveness_not_context  -0.4381     1.4037  -0.312 0.757780    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5735 on 23 degrees of freedom
## Multiple R-squared:  0.004217,   Adjusted R-squared:  -0.03908 
## F-statistic: 0.0974 on 1 and 23 DF,  p-value: 0.7578

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["effectiveness_not_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Statistical analysis of effectiveness not in context against PU

mdl <- lm(PU ~ effectiveness_not_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ effectiveness_not_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.01573 -0.21930  0.08645  0.12627  0.62627 
## 
## Coefficients:
##                           Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                 4.5015     0.7185   6.265 2.16e-06 ***
## effectiveness_not_context  -0.7424     0.9123  -0.814    0.424    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3727 on 23 degrees of freedom
## Multiple R-squared:  0.02799,    Adjusted R-squared:  -0.01427 
## F-statistic: 0.6623 on 1 and 23 DF,  p-value: 0.4241

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["effectiveness_not_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Relation with efficiency not in context

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = efficiency_not_context, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "efficiency not in context"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
   stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3, 3.25), label.x = c(0.15))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of efficiency against PEOU

mdl <- lm(PEOU ~  efficiency_not_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ efficiency_not_context, data = data_adt)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -1.3216 -0.3366  0.1506  0.3356  0.9463 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)               3.885      0.279  13.924 1.08e-12 ***
## efficiency_not_context    2.857      2.477   1.153    0.261    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5588 on 23 degrees of freedom
## Multiple R-squared:  0.05467,    Adjusted R-squared:  0.01357 
## F-statistic:  1.33 on 1 and 23 DF,  p-value: 0.2606

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["efficiency_not_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Statistical analysis of efficiency against PU

mdl <- lm(PU ~  efficiency_not_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ efficiency_not_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.99301 -0.19730  0.07241  0.13850  0.64475 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)              3.9430     0.1887  20.895   <2e-16 ***
## efficiency_not_context  -0.2231     1.6755  -0.133    0.895    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3779 on 23 degrees of freedom
## Multiple R-squared:  0.0007704,  Adjusted R-squared:  -0.04267 
## F-statistic: 0.01773 on 1 and 23 DF,  p-value: 0.8952

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["efficiency_not_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Relation with effectiveness in context

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = effectiveness_in_context, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "effectiveness in context"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
  stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(2, 2.25), label.x = c(0.6))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of effectiveness in context against PEOU

mdl <- lm(PEOU ~ effectiveness_in_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ effectiveness_in_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.27873 -0.15373 -0.02873  0.22127  0.84627 
## 
## Coefficients:
##                          Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                2.8356     0.5498   5.158 3.16e-05 ***
## effectiveness_in_context   1.4827     0.5958   2.488   0.0205 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5101 on 23 degrees of freedom
## Multiple R-squared:  0.2121, Adjusted R-squared:  0.1779 
## F-statistic: 6.192 on 1 and 23 DF,  p-value: 0.02051

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["effectiveness_in_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is rejected (p < 0.05 )

Statistical analysis of effectiveness not in context against PU

mdl <- lm(PU ~ effectiveness_in_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ effectiveness_in_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.98338 -0.12638  0.04016  0.13422  0.61116 
## 
## Coefficients:
##                          Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                3.5323     0.3991   8.852 7.25e-09 ***
## effectiveness_in_context   0.4275     0.4325   0.989    0.333    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3703 on 23 degrees of freedom
## Multiple R-squared:  0.04076,    Adjusted R-squared:  -0.0009505 
## F-statistic: 0.9772 on 1 and 23 DF,  p-value: 0.3332

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["effectiveness_in_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Relation with efficiency not in context

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = efficiency_in_context, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "efficiency in context"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
   stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3.2, 3.35), label.x = c(0.1))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of efficiency against PEOU

mdl <- lm(PEOU ~  efficiency_in_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ efficiency_in_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.09069 -0.30635  0.08505  0.25332  1.03431 
## 
## Coefficients:
##                       Estimate Std. Error t value Pr(>|t|)    
## (Intercept)             3.8986     0.2479   15.73 8.45e-14 ***
## efficiency_in_context   1.0656     0.8390    1.27    0.217    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5555 on 23 degrees of freedom
## Multiple R-squared:  0.06554,    Adjusted R-squared:  0.02491 
## F-statistic: 1.613 on 1 and 23 DF,  p-value: 0.2168

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["efficiency_in_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Statistical analysis of efficiency against PU

mdl <- lm(PU ~  efficiency_in_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ efficiency_in_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.00956 -0.17715  0.07778  0.13725  0.64077 
## 
## Coefficients:
##                       Estimate Std. Error t value Pr(>|t|)    
## (Intercept)             3.9623     0.1684  23.530   <2e-16 ***
## efficiency_in_context  -0.1602     0.5699  -0.281    0.781    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3774 on 23 degrees of freedom
## Multiple R-squared:  0.003423,   Adjusted R-squared:  -0.03991 
## F-statistic: 0.07899 on 1 and 23 DF,  p-value: 0.7812

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["efficiency_in_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Relation with transferability effectiveness

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = transferability_effectiveness, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "transferability effectiveness"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
  stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3, 3.25), label.x = c(0.6))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of effectiveness in context against PEOU

mdl <- lm(PEOU ~ transferability_effectiveness, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ transferability_effectiveness, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.13235 -0.34431  0.03069  0.19917  1.27055 
## 
## Coefficients:
##                               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                     3.7295     0.2749  13.567 1.84e-12 ***
## transferability_effectiveness   0.7344     0.4122   1.782    0.088 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5387 on 23 degrees of freedom
## Multiple R-squared:  0.1213, Adjusted R-squared:  0.08307 
## F-statistic: 3.174 on 1 and 23 DF,  p-value: 0.08802

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["transferability_effectiveness", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Statistical analysis of transferability effectiveness not in context against PU

mdl <- lm(PU ~ transferability_effectiveness, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ transferability_effectiveness, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.61021 -0.23396  0.04682  0.18523  0.48004 
## 
## Coefficients:
##                               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                     3.5392     0.1725  20.515 2.77e-16 ***
## transferability_effectiveness   0.6206     0.2587   2.399   0.0249 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3381 on 23 degrees of freedom
## Multiple R-squared:  0.2002, Adjusted R-squared:  0.1654 
## F-statistic: 5.755 on 1 and 23 DF,  p-value: 0.02494

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["transferability_effectiveness", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is rejected (p < 0.05 )

Relation with transferability efficiency

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = transferability_efficiency, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "transferability efficiency"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
   stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3, 3.25), label.x = c(0))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of transferability efficiency against PEOU

mdl <- lm(PEOU ~  transferability_efficiency, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ transferability_efficiency, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.11154 -0.22373  0.01616  0.24324  1.08321 
## 
## Coefficients:
##                            Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                  3.9168     0.2325  16.844 1.97e-14 ***
## transferability_efficiency  10.1080     7.8468   1.288     0.21    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.555 on 23 degrees of freedom
## Multiple R-squared:  0.06729,    Adjusted R-squared:  0.02674 
## F-statistic: 1.659 on 1 and 23 DF,  p-value: 0.2105

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["transferability_efficiency", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Statistical analysis of transferability efficiency against PU

mdl <- lm(PU ~  transferability_efficiency, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ transferability_efficiency, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.88956 -0.15237  0.04515  0.11615  0.62394 
## 
## Coefficients:
##                            Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                  3.8186     0.1565  24.392   <2e-16 ***
## transferability_efficiency   3.8941     5.2827   0.737    0.468    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3737 on 23 degrees of freedom
## Multiple R-squared:  0.02308,    Adjusted R-squared:  -0.01939 
## F-statistic: 0.5434 on 1 and 23 DF,  p-value: 0.4685

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["transferability_efficiency", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

ADT-Analysis

12-06-2023

Boxplots of effectiveness and efficiency variables

Boxplot of the three variables: PEOU, PU, ITU

Descriptive statistics for all the variables

Test of significance against average value 3 of PEOU, PU and ITU

PEOU and PU against efficiency and effectiveness, using a linear model as a fitting/interpolation function

Relation between PU and PEOU

Relation between ITU and PEOU; ITU and PU

Relation between PU and PEOU

Evaluation by single measures of effectiveness/efficiency

Relation with effectiveness not in context

Relation with efficiency not in context

Relation with effectiveness in context

Relation with efficiency not in context

Relation with transferability effectiveness

Relation with transferability efficiency