ADT-Analysis

library(ggplot2)
library(dplyr)
library(broom)
library(ggpubr)
library(tidyverse)
library(readr)
library(effsize)

## Warning: package 'effsize' was built under R version 4.3.2

data_adt <- read.csv("./ADT-test-results.csv", na.strings = c("N/A"))
data_adt <- data_adt %>% 
  filter(!(is.na(PEOU) | is.na(PU) | is.na(ITU))) %>%
    select(!ID)


SIZE = 18
EQ_SIZE = 6

PEU: perceived ease of use
PU: perceived usefulness
ITU: intention to use
Effectiveness: how high is the score of the subject
Efficiency: how fast is the subject

Boxplots of effectiveness and efficiency variables

Average total effectiveness is above 0.75, indicating a generally good level of comprehensibility of ADT principles. Furthermore, the boxplot shows that effectiveness in context is provides the highest contribution, followed by effectiveness not in context and transferability effectiveness. This suggests that while participants understand ADT principles, they find more difficulties in applying them in practice.

For the efficiency variable, we see a similar situation to that observed for effectiveness, thereby confirming that ADT ``in action’’ is perceived as a more difficult task.

Wilcoxon Signed Rank Test for efficiency and effectiveness

We evaluate if the variables of effectiveness and efficiency are significantly higher than target values.

ks.test(data_adt$Effectiveness, "pnorm")

## 
##  Asymptotic one-sample Kolmogorov-Smirnov test
## 
## data:  data_adt$Effectiveness
## D = 0.66747, p-value = 4.235e-10
## alternative hypothesis: two-sided

ks.test(data_adt$Efficiency, "pnorm")

## 
##  Asymptotic one-sample Kolmogorov-Smirnov test
## 
## data:  data_adt$Efficiency
## D = 0.50439, p-value = 5.979e-06
## alternative hypothesis: two-sided

Data are not normally distributed, therefore we use a nonparametric test (Wicoxon signed rank).

# Set the desired significance level
alpha <- 0.05

result <- wilcox.test(data_adt$Effectiveness, mu = 0.6, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$Effectiveness
## V = 298, p-value = 0.000139
## alternative hypothesis: true location is greater than 0.6

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that Effectiveness is lower than 0.6 is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis that Effectiveness is lower than 0.6 is rejected (p < 0.05 )

# Set the desired significance level
alpha <- 0.05

result <- wilcox.test(data_adt$Efficiency, mu = 0.015, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$Efficiency
## V = 324, p-value = 7.381e-06
## alternative hypothesis: true location is greater than 0.015

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that Efficiency is lower than 0.6 is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis that Efficiency is lower than 0.6 is rejected (p < 0.05 )

Both null hypotheses about efficiency and effectiveness are rejected.

Wilcoxon signed rank for efficiency and effectiveness (detailed variables):

effectiveness_not_context: rejected

# Set the desired significance level
alpha <- 0.05

result <- wilcox.test(data_adt$effectiveness_not_context, mu = 0.6, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$effectiveness_not_context
## V = 325, p-value = 6.017e-06
## alternative hypothesis: true location is greater than 0.6

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that effectiveness_not_context is lower than 0.6 is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis that effectiveness_not_context is lower than 0.6 is rejected (p < 0.05 )

efficiency_not_context: rejected

# Set the desired significance level
alpha <- 0.05

result <- wilcox.test(data_adt$efficiency_not_context, mu = 0.06, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$efficiency_not_context
## V = 297.5, p-value = 0.0001476
## alternative hypothesis: true location is greater than 0.06

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that efficiency_not_context is lower than 0.6 is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis that efficiency_not_context is lower than 0.6 is rejected (p < 0.05 )

transferability_effectiveness: not rejected (p-value = 0.1121)

# Set the desired significance level
alpha <- 0.05

result <- wilcox.test(data_adt$transferability_effectiveness, mu = 0.6, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$transferability_effectiveness
## V = 208, p-value = 0.1121
## alternative hypothesis: true location is greater than 0.6

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that transferability_effectiveness is lower than 0.6 is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

transferability_efficiency: not rejected (p-value= 0.2295)

# Set the desired significance level
alpha <- 0.05

result <- wilcox.test(data_adt$transferability_efficiency, mu = 0.024, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$transferability_efficiency
## V = 190.5, p-value = 0.2295
## alternative hypothesis: true location is greater than 0.024

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that transferability_efficiency is lower than 0.6 is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

effectiveness_in_context: rejected

# Set the desired significance level
alpha <- 0.05

result <- wilcox.test(data_adt$effectiveness_in_context, mu = 0.6, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$effectiveness_in_context
## V = 300, p-value = 7.603e-05
## alternative hypothesis: true location is greater than 0.6

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that effectiveness_in_context is lower than 0.6 is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis that effectiveness_in_context is lower than 0.6 is rejected (p < 0.05 )

efficiency_in_context: rejected

# Set the desired significance level
alpha <- 0.05

result <- wilcox.test(data_adt$efficiency_in_context, mu = 0.12, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$efficiency_in_context
## V = 309, p-value = 4.191e-05
## alternative hypothesis: true location is greater than 0.12

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that efficiency_in_context is lower than 0.6 is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis that efficiency_in_context is lower than 0.6 is rejected (p < 0.05 )

Effect size for the variables

Reference values for effect size are:

A value of 0.2 represents a small effect size.
A value of 0.5 represents a medium effect size.
A value of 0.8 represents a large effect size.

library(lsr)

## Warning: package 'lsr' was built under R version 4.3.2

cohensD(data_adt$Effectiveness, mu = 0.6)

## [1] 1.255714

cohensD(data_adt$Efficiency, mu = 0.015)

## [1] 1.983621

The effect size for effectiveness and efficiency is small.

cohensD(data_adt$effectiveness_not_context, mu = 0.6)

## [1] 2.19877

cohensD(data_adt$efficiency_not_context, mu = 0.06)

## [1] 0.9383547

cohensD(data_adt$transferability_effectiveness, mu = 0.6)

## [1] 0.05053103

cohensD(data_adt$transferability_efficiency, mu = 0.024)

## [1] 0.1412935

cohensD(data_adt$effectiveness_in_context, mu = 0.6)

## [1] 1.755117

cohensD(data_adt$efficiency_in_context, mu = 0.12)

## [1] 1.066268

The effect size for detailed variables is as follows (note that effect size makes sense only for significant values, i.e., when the hypothesis is rejected).

Effectiveness not in context: small
Efficiency not in context: no effect
Transferability effectiveness: no effect
Transferability efficiency: small
Effectiveness in context: small
Efficiency in context: small

Boxplot of the three variables: PEOU, PU, ITU

The three variables are clearly all above the average value of the Likert scale (i.e., 3), thus suggesting that the general degree of acceptance of the ADT notation is high. We see that, while ITU and PU have comparable values, PEOU receives the highest score. This further confirms that ADTs are considered easy to use by the participants.

Descriptive statistics for all the variables

df <- data_adt
col_names <- c("Stat", colnames(df))
# Create a vector with the desired summary statistics
stats <- c("Min", "1st Qu", "Median", "Mean", "3rd Qu", "Max")

# Create an empty matrix to store the summary statistics
summary_mat <- matrix(NA, nrow = length(stats), ncol = ncol(df))

# Iterate over each column of the original dataframe
for (i in 1:ncol(df)) {
  # Compute the summary statistics
  summary_stats <- summary(df[[i]])
  
  # Store the summary statistics in the matrix
  summary_mat[, i] <- summary_stats
}

# Convert the matrix to a dataframe
summary_df <- as.data.frame(summary_mat)

# Add a row with the mean of each column
#summary_df <- rbind(summary_df, colMeans(df))

# Rename the row names for better readability
row.names(summary_df) <- c(stats)

colnames(summary_df) <- c("PEOU", "PU", "ITU", "Effectiveness", "Efficiency", "Effv. not in Context", "Effc. not in Context", "Transferability Effv.", "Transferability Effc.", "Effv. in Context", "Effc. in Context")

# Transpose the summary dataframe
transposed_df <- t(summary_df)

# Print the transposed dataframe
View(transposed_df)

knitr::kable(transposed_df)

	Min	1st Qu	Median	Mean	3rd Qu	Max
PEOU	2.875	4.000	4.250	4.18000	4.500	5.000
PU	2.929	3.714	4.000	3.91996	4.071	4.571
ITU	3.000	3.625	3.875	3.88000	4.125	5.000
Effectiveness	0.287	0.769	0.792	0.76780	0.833	0.986
Efficiency	0.011	0.101	0.118	0.13116	0.172	0.241
Effv. not in Context	0.625	0.750	0.750	0.78336	0.833	0.958
Effc. not in Context	0.024	0.069	0.094	0.10320	0.150	0.188
Transferability Effv.	0.000	0.556	0.667	0.61348	0.778	1.000
Transferability Effc.	0.000	0.018	0.023	0.02604	0.036	0.049
Effv. in Context	0.111	0.889	0.889	0.90672	1.000	1.000
Effc. in Context	0.009	0.178	0.250	0.26412	0.333	0.500

Test of significance against average value 3 of PEOU, PU and ITU

The normality check, performed with the Kolmogorov-Smirnov test, fails for all the variables with p-value well below the 0.05 significance level. We therefore need to apply a nonparametric test(Wicoxon signed rank) to check whether the three variables are above the average value of the Livert scale (i.e., 3).

ks.test(data_adt$PEOU, "pnorm")

## 
##  Asymptotic one-sample Kolmogorov-Smirnov test
## 
## data:  data_adt$PEOU
## D = 0.99798, p-value < 2.2e-16
## alternative hypothesis: two-sided

ks.test(data_adt$PU, "pnorm")

## 
##  Asymptotic one-sample Kolmogorov-Smirnov test
## 
## data:  data_adt$PU
## D = 0.9983, p-value < 2.2e-16
## alternative hypothesis: two-sided

ks.test(data_adt$ITU, "pnorm")

## 
##  Asymptotic one-sample Kolmogorov-Smirnov test
## 
## data:  data_adt$ITU
## D = 0.99865, p-value < 2.2e-16
## alternative hypothesis: two-sided

Data are not normally distributed, therefore we use a nonparametric test (Wicoxon signed rank).

# Set the desired significance level
alpha <- 0.05

result <- wilcox.test(data_adt$PEOU, mu = 3, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$PEOU
## V = 299, p-value = 1.077e-05
## alternative hypothesis: true location is greater than 3

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that PEOU is lower than the average score is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis that PEOU is lower than the average score is rejected (p < 0.05 )

result <- wilcox.test(data_adt$PU, mu = 3, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$PU
## V = 324, p-value = 7.109e-06
## alternative hypothesis: true location is greater than 3

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that PU is lower than the average score is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis that PU is lower than the average score is rejected (p < 0.05 )

result <- wilcox.test(data_adt$ITU, mu = 3, alternative = "greater")
result

## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  data_adt$ITU
## V = 300, p-value = 9.282e-06
## alternative hypothesis: true location is greater than 3

# Check if the p-value is lower than the significance level
if (result$p.value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis that ITU is lower than the average score is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis that ITU is lower than the average score is rejected (p < 0.05 )

For all the tests we can reject the NULL hypothesis. More specifically, we reject the following hypotheses.

ADT is perceived as difficult to use
ADT is perceived as not useful
There is no intention to use ADT

Effect size for the three variables

Effect size for the variables

library(lsr)

cohensD(data_adt$PEOU, mu = 3)

## [1] 2.097433

cohensD(data_adt$PU, mu = 3)

## [1] 2.485847

cohensD(data_adt$ITU, mu = 3)

## [1] 2.185815

The effect size is medium for PEOU and large for PU and ITU.

PEOU and PU against efficiency and effectiveness, using a linear model as a fitting/interpolation function

We now explore the relation with total effectiveness.

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = Effectiveness, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "effectiveness"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
  stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(2.5, 2.75), label.x = c(0.6))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of effectiveness against PEOU

mdl <- lm(PEOU ~ Effectiveness, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ Effectiveness, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.08582 -0.30712  0.02282  0.23528  1.21186 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)     2.8235     0.6206   4.549 0.000143 ***
## Effectiveness   1.7667     0.7968   2.217 0.036772 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5216 on 23 degrees of freedom
## Multiple R-squared:  0.1761, Adjusted R-squared:  0.1403 
## F-statistic: 4.916 on 1 and 23 DF,  p-value: 0.03677

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["Effectiveness", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is rejected (p < 0.05 )

Statistical analysis of effectiveness against PU

mdl <- lm(PU ~ Effectiveness, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ Effectiveness, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.77501 -0.14883  0.03019  0.13340  0.55648 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)     3.1724     0.4211   7.534 1.18e-07 ***
## Effectiveness   0.9736     0.5406   1.801   0.0848 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3539 on 23 degrees of freedom
## Multiple R-squared:  0.1236, Adjusted R-squared:  0.08549 
## F-statistic: 3.244 on 1 and 23 DF,  p-value: 0.08483

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["Effectiveness", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

From the plot, we see that when effectiveness grows also PEOU and PU grows. From the statistical analysis, however, we see that while the relationship between effectiveness and PEOU is significant, the one with PU is not. Therefore the first NULL hypothesis is rejected, while the second NULL hypothesis cannot be rejected.

PEOU will not be determined by effectiveness: reject
PU will not be determined by effectiveness: fail to reject

Relation with efficiency

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = Efficiency, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "efficiency"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
   stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3, 3.25), label.x = c(0.15))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of efficiency against PEOU

mdl <- lm(PEOU ~  Efficiency, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ Efficiency, data = data_adt)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -1.1082 -0.3505  0.1054  0.2439  1.0464 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   3.8272     0.2753  13.901 1.11e-12 ***
## Efficiency    2.6902     1.9231   1.399    0.175    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5517 on 23 degrees of freedom
## Multiple R-squared:  0.07841,    Adjusted R-squared:  0.03834 
## F-statistic: 1.957 on 1 and 23 DF,  p-value: 0.1752

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["Efficiency", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Statistical analysis of efficiency against PU

mdl <- lm(PU ~  Efficiency, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ Efficiency, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.00366 -0.18575  0.07646  0.13986  0.64391 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   3.9532     0.1885  20.972   <2e-16 ***
## Efficiency   -0.2532     1.3167  -0.192    0.849    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3777 on 23 degrees of freedom
## Multiple R-squared:  0.001605,   Adjusted R-squared:  -0.0418 
## F-statistic: 0.03697 on 1 and 23 DF,  p-value: 0.8492

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["Efficiency", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

From the plot, we see that while PEOU grows with efficiency, PU appears to slightly decrease. From the statistical analysis, however, we see that none of the relationships are significant. Therefore none of the NULL hypothesis can be rejected.

PEOU will not be determined by efficiency: fail reject
PU will not be determined by efficiency: fail to reject

Relation between PU and PEOU

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of PU against PEOU

mdl <- lm(PU ~ PEOU, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ PEOU, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.05101 -0.12078  0.06576  0.13676  0.64591 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  3.61384    0.57471   6.288 2.04e-06 ***
## PEOU         0.07323    0.13631   0.537    0.596    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3757 on 23 degrees of freedom
## Multiple R-squared:  0.01239,    Adjusted R-squared:  -0.03054 
## F-statistic: 0.2886 on 1 and 23 DF,  p-value: 0.5962

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["PEOU", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

We see that although there is a slight increase of PU when PEOU grows, the relationship has no statistical significance. Therefore:

PU will not be determined by PEOU: fail to reject

Relation between ITU and PEOU; ITU and PU

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = value, y = ITU, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "indicator"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
  stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3, 3.25), label.x = c(3.5))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of PU against ITU

mdl <- lm(ITU ~ PU, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = ITU ~ PU, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.33404 -0.19893 -0.07393  0.17422  0.55933 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   0.5041     0.5456   0.924    0.365    
## PU            0.8612     0.1386   6.214 2.44e-06 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.2513 on 23 degrees of freedom
## Multiple R-squared:  0.6267, Adjusted R-squared:  0.6105 
## F-statistic: 38.61 on 1 and 23 DF,  p-value: 2.436e-06

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["PU", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is rejected (p < 0.05 )

Statistical analysis of PEOU against ITU

mdl <- lm(ITU ~ PEOU, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = ITU ~ PEOU, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.82323 -0.17595  0.03742  0.19187  1.10351 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   2.8950     0.5940   4.874 6.38e-05 ***
## PEOU          0.2356     0.1409   1.672    0.108    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3883 on 23 degrees of freedom
## Multiple R-squared:  0.1084, Adjusted R-squared:  0.06967 
## F-statistic: 2.797 on 1 and 23 DF,  p-value: 0.108

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["PEOU", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

We clearly see from the plot above that both PEOU and and PU tend to grow with ITU. However, from the analysis below, we see that PEOU and PU do not have any relationship. Therefore:

PU will not be determined by PEOU: fail to reject
ITU will not be determined by PEOU: fail to reject
ITU will not be determined by PU: reject

Relation between PU and PEOU

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of PU against PEOU

mdl <- lm(PU ~ PEOU, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ PEOU, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.05101 -0.12078  0.06576  0.13676  0.64591 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  3.61384    0.57471   6.288 2.04e-06 ***
## PEOU         0.07323    0.13631   0.537    0.596    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3757 on 23 degrees of freedom
## Multiple R-squared:  0.01239,    Adjusted R-squared:  -0.03054 
## F-statistic: 0.2886 on 1 and 23 DF,  p-value: 0.5962

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["PEOU", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Evaluation by single measures of effectiveness/efficiency

PEOU will not be determined by effectiveness not in context: fail to reject
PU will not be determined by effectiveness not in context: fail to reject
PEOU will not be determined by efficiency not in context: fail to reject
PU will not be determined by efficiency not in context: fail to reject
PEOU will not be determined by effectiveness in context: reject
PU will not be determined by effectiveness in context: fail to reject
PEOU will not be determined by efficiency in context: fail to reject
PU will not be determined by efficiency in context: fail to reject
PEOU will not be determined by transferability effectiveness: fail to reject
PU will not be determined by transferability effectiveness: reject
PEOU will not be determined by transferability efficiency: fail to reject
PU will not be determined by transferability efficiency: fail to reject

Relation with effectiveness not in context

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = effectiveness_not_context, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "effectiveness not in context"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
  stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3, 3.25), label.x = c(0.6))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of effectiveness not in context against PEOU

mdl <- lm(PEOU ~ effectiveness_not_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ effectiveness_not_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.26485 -0.24938  0.05538  0.26902  0.89651 
## 
## Coefficients:
##                           Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                 4.5232     1.1056   4.091 0.000448 ***
## effectiveness_not_context  -0.4381     1.4037  -0.312 0.757780    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5735 on 23 degrees of freedom
## Multiple R-squared:  0.004217,   Adjusted R-squared:  -0.03908 
## F-statistic: 0.0974 on 1 and 23 DF,  p-value: 0.7578

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["effectiveness_not_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Statistical analysis of effectiveness not in context against PU

mdl <- lm(PU ~ effectiveness_not_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ effectiveness_not_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.01573 -0.21930  0.08645  0.12627  0.62627 
## 
## Coefficients:
##                           Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                 4.5015     0.7185   6.265 2.16e-06 ***
## effectiveness_not_context  -0.7424     0.9123  -0.814    0.424    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3727 on 23 degrees of freedom
## Multiple R-squared:  0.02799,    Adjusted R-squared:  -0.01427 
## F-statistic: 0.6623 on 1 and 23 DF,  p-value: 0.4241

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["effectiveness_not_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Relation with efficiency not in context

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = efficiency_not_context, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "efficiency not in context"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
   stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3, 3.25), label.x = c(0.15))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of efficiency against PEOU

mdl <- lm(PEOU ~  efficiency_not_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ efficiency_not_context, data = data_adt)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -1.3216 -0.3366  0.1506  0.3356  0.9463 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)               3.885      0.279  13.924 1.08e-12 ***
## efficiency_not_context    2.857      2.477   1.153    0.261    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5588 on 23 degrees of freedom
## Multiple R-squared:  0.05467,    Adjusted R-squared:  0.01357 
## F-statistic:  1.33 on 1 and 23 DF,  p-value: 0.2606

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["efficiency_not_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Statistical analysis of efficiency against PU

mdl <- lm(PU ~  efficiency_not_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ efficiency_not_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.99301 -0.19730  0.07241  0.13850  0.64475 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)              3.9430     0.1887  20.895   <2e-16 ***
## efficiency_not_context  -0.2231     1.6755  -0.133    0.895    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3779 on 23 degrees of freedom
## Multiple R-squared:  0.0007704,  Adjusted R-squared:  -0.04267 
## F-statistic: 0.01773 on 1 and 23 DF,  p-value: 0.8952

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["efficiency_not_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Relation with effectiveness in context

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = effectiveness_in_context, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "effectiveness in context"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
  stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(2, 2.25), label.x = c(0.6))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of effectiveness in context against PEOU

mdl <- lm(PEOU ~ effectiveness_in_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ effectiveness_in_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.27873 -0.15373 -0.02873  0.22127  0.84627 
## 
## Coefficients:
##                          Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                2.8356     0.5498   5.158 3.16e-05 ***
## effectiveness_in_context   1.4827     0.5958   2.488   0.0205 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5101 on 23 degrees of freedom
## Multiple R-squared:  0.2121, Adjusted R-squared:  0.1779 
## F-statistic: 6.192 on 1 and 23 DF,  p-value: 0.02051

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["effectiveness_in_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is rejected (p < 0.05 )

Statistical analysis of effectiveness not in context against PU

mdl <- lm(PU ~ effectiveness_in_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ effectiveness_in_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.98338 -0.12638  0.04016  0.13422  0.61116 
## 
## Coefficients:
##                          Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                3.5323     0.3991   8.852 7.25e-09 ***
## effectiveness_in_context   0.4275     0.4325   0.989    0.333    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3703 on 23 degrees of freedom
## Multiple R-squared:  0.04076,    Adjusted R-squared:  -0.0009505 
## F-statistic: 0.9772 on 1 and 23 DF,  p-value: 0.3332

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["effectiveness_in_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Relation with efficiency not in context

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = efficiency_in_context, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "efficiency in context"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
   stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3.2, 3.35), label.x = c(0.1))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of efficiency against PEOU

mdl <- lm(PEOU ~  efficiency_in_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ efficiency_in_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.09069 -0.30635  0.08505  0.25332  1.03431 
## 
## Coefficients:
##                       Estimate Std. Error t value Pr(>|t|)    
## (Intercept)             3.8986     0.2479   15.73 8.45e-14 ***
## efficiency_in_context   1.0656     0.8390    1.27    0.217    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5555 on 23 degrees of freedom
## Multiple R-squared:  0.06554,    Adjusted R-squared:  0.02491 
## F-statistic: 1.613 on 1 and 23 DF,  p-value: 0.2168

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["efficiency_in_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Statistical analysis of efficiency against PU

mdl <- lm(PU ~  efficiency_in_context, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ efficiency_in_context, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.00956 -0.17715  0.07778  0.13725  0.64077 
## 
## Coefficients:
##                       Estimate Std. Error t value Pr(>|t|)    
## (Intercept)             3.9623     0.1684  23.530   <2e-16 ***
## efficiency_in_context  -0.1602     0.5699  -0.281    0.781    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3774 on 23 degrees of freedom
## Multiple R-squared:  0.003423,   Adjusted R-squared:  -0.03991 
## F-statistic: 0.07899 on 1 and 23 DF,  p-value: 0.7812

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["efficiency_in_context", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Relation with transferability effectiveness

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = transferability_effectiveness, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "transferability effectiveness"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
  stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3, 3.25), label.x = c(0.6))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of effectiveness in context against PEOU

mdl <- lm(PEOU ~ transferability_effectiveness, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ transferability_effectiveness, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.13235 -0.34431  0.03069  0.19917  1.27055 
## 
## Coefficients:
##                               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                     3.7295     0.2749  13.567 1.84e-12 ***
## transferability_effectiveness   0.7344     0.4122   1.782    0.088 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5387 on 23 degrees of freedom
## Multiple R-squared:  0.1213, Adjusted R-squared:  0.08307 
## F-statistic: 3.174 on 1 and 23 DF,  p-value: 0.08802

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["transferability_effectiveness", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Statistical analysis of transferability effectiveness not in context against PU

mdl <- lm(PU ~ transferability_effectiveness, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ transferability_effectiveness, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.61021 -0.23396  0.04682  0.18523  0.48004 
## 
## Coefficients:
##                               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                     3.5392     0.1725  20.515 2.77e-16 ***
## transferability_effectiveness   0.6206     0.2587   2.399   0.0249 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3381 on 23 degrees of freedom
## Multiple R-squared:  0.2002, Adjusted R-squared:  0.1654 
## F-statistic: 5.755 on 1 and 23 DF,  p-value: 0.02494

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["transferability_effectiveness", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is rejected (p < 0.05 )

Relation with transferability efficiency

data_adt %>%  
  pivot_longer(
  cols = c("PEOU", "PU"),
  names_to = "indicator",
  values_to = "value"
  ) %>% 
  ggplot(aes(x = transferability_efficiency, y = value, color = indicator)) +
  geom_smooth(method = lm) +
  labs(
    x = "transferability efficiency"
  ) +
  theme(axis.text = element_text(size = SIZE),
        axis.title = element_text(size = SIZE),
        legend.text = element_text(size = SIZE),
        legend.title = element_text(size = SIZE)) +
   stat_regline_equation(size=EQ_SIZE, show.legend = FALSE, label.y = c(3, 3.25), label.x = c(0))

## `geom_smooth()` using formula = 'y ~ x'

Statistical analysis of transferability efficiency against PEOU

mdl <- lm(PEOU ~  transferability_efficiency, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PEOU ~ transferability_efficiency, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.11154 -0.22373  0.01616  0.24324  1.08321 
## 
## Coefficients:
##                            Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                  3.9168     0.2325  16.844 1.97e-14 ***
## transferability_efficiency  10.1080     7.8468   1.288     0.21    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.555 on 23 degrees of freedom
## Multiple R-squared:  0.06729,    Adjusted R-squared:  0.02674 
## F-statistic: 1.659 on 1 and 23 DF,  p-value: 0.2105

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["transferability_efficiency", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

Statistical analysis of transferability efficiency against PU

mdl <- lm(PU ~  transferability_efficiency, data = data_adt)
summary(mdl)

## 
## Call:
## lm(formula = PU ~ transferability_efficiency, data = data_adt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.88956 -0.15237  0.04515  0.11615  0.62394 
## 
## Coefficients:
##                            Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                  3.8186     0.1565  24.392   <2e-16 ***
## transferability_efficiency   3.8941     5.2827   0.737    0.468    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.3737 on 23 degrees of freedom
## Multiple R-squared:  0.02308,    Adjusted R-squared:  -0.01939 
## F-statistic: 0.5434 on 1 and 23 DF,  p-value: 0.4685

# Get the summary of the model
mdl_summary <- summary(mdl)

# Extract the p-value
p_value <- mdl_summary$coefficients["transferability_efficiency", "Pr(>|t|)"]

# Set the desired significance level
alpha <- 0.05

# Check if the p-value is lower than the significance level
if (p_value < alpha) {
  # Null hypothesis rejected
  cat("The null hypothesis is rejected (p <", alpha, ")")
} else {
  # Null hypothesis not rejected
  cat("The null hypothesis is not rejected (p >=", alpha, ")")
}

## The null hypothesis is not rejected (p >= 0.05 )

ADT-Analysis

12-06-2023

Boxplots of effectiveness and efficiency variables

Wilcoxon Signed Rank Test for efficiency and effectiveness

Wilcoxon signed rank for efficiency and effectiveness (detailed variables):

Effect size for the variables

Boxplot of the three variables: PEOU, PU, ITU

Descriptive statistics for all the variables

Test of significance against average value 3 of PEOU, PU and ITU

Effect size for the three variables

Effect size for the variables

PEOU and PU against efficiency and effectiveness, using a linear model as a fitting/interpolation function

Relation between PU and PEOU

Relation between ITU and PEOU; ITU and PU

Relation between PU and PEOU

Evaluation by single measures of effectiveness/efficiency

Relation with effectiveness not in context

Relation with efficiency not in context

Relation with effectiveness in context

Relation with efficiency not in context

Relation with transferability effectiveness

Relation with transferability efficiency