Published June 9, 2021 | Version v1
Journal article Open

Complex Thinking and Sustainable Social Development: Validity and Reliability of the COMPLEX-21 Scale

Authors/Creators

  • 1. University Center CIFE

Description

Thinking skills are essential to achieve sustainable social development. Nonetheless, there is no specific instrument that assesses all of these skills as a whole. The present study aimed to design and validate a scale to assess complex thinking skills in adult people. A scale of 22 items assessing the following aspects: analysis and problem solving, critical analysis, metacognition, systemic analysis, and creativity, in five levels, was created. This scale was validated in 626 university students from Peru. In total, 16 experts in the field helped to determine the content validity of the scale (Aiken’s V value higher than 0.8). The confirmatory factor analysis allowed the evaluation of the structure of the five factors theoretically proposed and the goodness of fit indexes was satisfactory. An item was eliminated during the process and the scale resulted in 21 items. The composite reliability for the different factors was ranged between 0.794 and 0.867. The invariance between genders was also checked and the concurrent validity was proved. The study concludes that the content validity, construct validity, concurrent validity, and composite reliability levels of the COMPLEX-21 scale are appropriate

Files

sustainability-13-06591-v3.pdf

Files (721.7 kB)

Name Size Download all
md5:129764808a5a53e2fa9ec5c7ca1e421b
721.7 kB Preview Download

Additional details

References

  • Mylek, M.R.; Schirmer, J. Understanding acceptability of fuel management to reduce wildfire risk: Informing communication through understanding complexity of thinking. For. Policy Econ. 2020, 113, 102120, doi:10.1016/j.forpol.2020.102120.
  • Axpe, M.R.V. Ciencias de la complejidad vs. pensamiento complejo. Claves para una lectura crítica del concepto de cientifi‐ cidad. En Reynoso, C. Pensam. Rev. Investig. Inf. Filos. 2019, 75, 87–106.
  • Costa, V.T.; Meirelles, B.H.S. Adherence to treatment of young adults living with HIV/AIDS from the perspective of complex thinking. Texto Context. Enferm. 2019, 28, 1–15, doi:10.1590/1980‐265x‐tce‐2017‐0016.
  • Stoian, A.P.; Mitrofan, G.; Colceag, F.; Suceveanu, A.I.; Hainarosie, R.; Pituru, S.; Diaconu, C.C.; Timofte, D.; Nitipir, C.; Poiana, C.; et al. Oxidative Stress in Diabetes. A model of complex thinking applied in medicine. Rev. Chim. 2018, 69, 2515–2519, doi:10.37358/rc.18.9.6566.
  • Block, M. Complex Environment Calls for Complex Thinking: About Knowledge Sharing Culture. J. Rev. Glob. Econ. 2019, 8, 141–152, doi:10.6000/1929‐7092.2019.08.14.
  • Lin, Y.‐T. Impacts of a flipped classroom with a smart learning diagnosis system on students' learning performance, perception, and problem solving ability in a software engineering course. Comput. Hum. Behav. 2019, 95, 187–196, doi:10.1016/j.chb.2018.11.036.
  • Krevetzakis, E. On the Centrality of Physical/Motor Activities in Primary Education. J. Adv. Educ. Res. 2019, 4, 24–33, doi:10.22606/jaer.2019.41003.
  • Hanlon, J.P.; Prihoda, T.J.; Verrett, R.G.; Jones, J.D.; Haney, S.J.; Hendricson, W.D. Critical Thinking in Dental Students and Experienced Practitioners Assessed by the Health Sciences Reasoning Test. J. Dent. Educ. 2018, 82, 916–920, doi:10.21815/jde.018.089
  • Peeters, M.J.; Zitko, K.L.; Schmude, K.A. Development of Critical Thinking in Pharmacy Education. Innov. Pharm. 2016, 7, doi:10.24926/iip.v7i1.415.
  • Tran, T.B.L.; Ho, T.N.; MacKenzie, S.V.; Le, L.K. Developing assessment criteria of a lesson for creativity to promote teaching for creativity. Think. Ski. Creat. 2017, 25, 10–26, doi:10.1016/j.tsc.2017.05.006.
  • Martinsen, Ø.L.; Furnham, A. Cognitive style and competence motivation in creative problem solving. Pers. Individ. Differ. 2019, 139, 241–246, doi:10.1016/j.paid.2018.11.023.
  • Morin, E. The Seven Knowledge Necessary for the Education of the Future; Santillana‐Unesco: Paris, France, 1999.
  • Morin, E. Introduction to Complex Thinking; Gedisa: Barcelona, Spain, 1995
  • Terrado, P.R. Aplicación de las teorías de la complejidad a la comprensión del territorio. Estud. Geogr. 2018, 79, 237–265, doi:10.3989/estgeogr.201810
  • Morin, E. La Mente Bien Ordenada. Repensar la Reforma. Reformar el Pensamiento; Seix Barral: Barcelona, Spain, 2000.
  • Lipman, M. Pensamiento Complejo y Educación [Complex Thinking and Education]; Ediciones de la Torre: Madrid, Spain, 1997.
  • Saremi, H.; Bahdori, S. The Relationship between Critical Thinking with Emotional Intelligence and Creativity among Elemen‐ tary School Principals in Bojnord City, Iran. Int. J. Life Sci. 2015, 9, 33–40, doi:10.3126/ijls.v9i6.12684.
  • Klimenko, O.; Aristizábal, A.; Restrepo, C. Pensamiento crítico y creativo en la educación preescolar: Algunos aportes desde la neuropsicopedagogia. Katharsis 2019, 28, 59–89, doi:10.25057/25005731.1258.
  • Hasan, R.; Lukitasari, M.; Utami, S.; Anizar, A. The activeness, critical, and creative thinking skills of students in the Lesson Study‐based inquiry and cooperative learning. J. Pendidik. Biol. Indones. 2019, 5, 77–84, doi:10.22219/jpbi.v5i1.7328.
  • Siburian, O.; Corebima, A.D.; Saptasari, M. The Correlation between Critical and Creative Thinking Skills on Cognitive Learning Results. Eurasian J. Educ. Res. 2019, 19, 1–16, doi:10.14689/ejer.2019.81.6.
  • Avetisyan, N.; Hayrapetyan, L.R. Mathlet as a new approach for improving critical and creative thinking skills in mathematics. Int. J. Educ. Res. 2017, 12. Available online: https://cutt.ly/otxYbSB (accessed on 8 May 2021).
  • Beaty, R.E.; Benedek, M.; Silvia, P.J.; Schacter, D.L. Creative Cognition and Brain Network Dynamics. Trends Cogn. Sci. 2016, 20, 87–95, doi:10.1016/j.tics.2015.10.004.
  • Asefi, M.; Imani, E. Effects of active strategic teaching model (ASTM) in creative and critical thinking skills of architecture students. Archnet‐IJAR Int. J. Arch. Res. 2018, 12, 209–222, doi:10.26687/archnet‐ijar.v12i2.1340.
  • Beaty, R.E.; Seli, P.; Schacter, D.L. Network neuroscience of creative cognition: Mapping cognitive mechanisms and individual differences in the creative brain. Curr. Opin. Behav. Sci. 2019, 27, 22–30, doi:10.1016/j.cobeha.2018.08.013.
  • Pacheco, C.S. Art Education for the Development of Complex Thinking Metacompetence: A Theoretical Approach. Int. J. Art Des. Educ. 2019, 39, 242–254, doi:10.1111/jade.12261.
  • Murrain, E.; Barrera, N.F.; Vargas, Y. Cuatro reflexiones sobre la docencia. Rev. Repert. Med. Cirugía 2017, 26, 242–248, doi:10.1016/j.reper.2017.09.001.
  • Puziol, J.K.P.; Barreyro, G.B. Alfa tuning Latin America project: The relationship between elaboration and imple‐mentation in participating Brazilian universities. Acta Sci. 2018, 40, e37338.
  • Gomez, J.T.A. La competencia europeísta: Una competencia integradora. Bordón. Rev. Pedagog. 2015, 67, 35, doi:10.13042/bor‐ don.2015.67302.
  • Cuadra‐Martínez, D.J.; Castro, P.J.; Juliá, M.T. Tres Saberes en la Formación Profesional por Competencias: Integración de Teo‐ rías Subjetivas, Profesionales y Científicas. Form. Univ. 2018, 11, 19–30, doi:10.4067/s0718‐50062018000500019.
  • Palma, M.; Rios, I.D.L.; Miñán, E. Generic competences in engineering field: A comparative study between Latin America and European Union. Procedia Soc. Behav. Sci. 2011, 15, 576–585, doi:10.1016/j.sbspro.2011.03.144.
  • Beneitone, P.; Esquetini, C.; González, J.; Maletá, M.M.; Siufi, G.; Wagenaar, R. Reflexões e Perspectivas da Edu‐Cação Superior na América Latina: Informe Final do Projeto Tuning—2004–2007; Universidad de Deusto: Bilbao, Spain, 2007.
  • Anastacio, M.R. Proposals for Teacher Training in the Face of the Challenge of Educating for Sustainable Development: Beyond Epistemologies and Methodologies. In Universities and Sustainable Communities: Meeting the Goals of the Agenda 2030; Springer: Cham, Switzerland, 2020.
  • Luna‐Nemecio, J.; Tobón, S.; Juárez‐Hernández, L.G. Sustainability‐based on socioformation and complex thought or sustaina‐ ble social development. Resour. Environ. Sustain. 2020, 2, 100007, doi:10.1016/j.resenv.2020.100007.
  • Luna‐Nemecio, J. Para pensar el Desarrollo Social Sostenible: Múltiples Enfoques, un Mismo Objetivo. 2020. Available online: https://www.researchgate.net/profile/Josemanuel‐Luna‐Nemecio/publication/339628029_Para_pensar_el_desarrollo_so‐ cial_sostenible_multiples_enfoques_un_mismo_objetivo/links/5e5d2bfaa6fdccbeba138607/Para‐pensar‐el‐desarrollo‐social‐ sostenible‐multiples‐enfoques‐un‐mismo‐objetivo.pdf (accessed on 8 May 2021).
  • Jarquín‐Cisneros, L.M. How to generate ethical teachers through Socioformation to achieve Sustainable Social Development? Ecocience Int. J. 2019, 1, 29–32, doi:10.35766/je19114
  • Santoyo‐Ledesma, D. Approach to Sustainable Social Development and Human Talent Management in the context of sociofor‐ mation. Ecocience Int. J. 2019, 1, 112–123, doi:10.35766/je191112
  • Maury Mena, S.C.; Marín Escobar, J.C.; Ortiz Padilla, M.; Gravini Donado, M. Competencias genéricas en estudiantes de edu‐ cación superior de una universidad privada de Barranquilla Colombia, desde la perspectiva del Proyecto Alfa Tuning América Latina y del Ministerio de Educación Nacional de Colombia (MEN). Rev. Espac. 2018, 39. Available online: https://cutt.ly/vtnRKmb (accessed on 8 May 2021).
  • Batrićević, A.; Joldžić, V.; Stanković, V.; Paunović, N. Solving the Problems of Rural as Environmentally Desirable Segment of Sustainable Development. Econ. Anal. 2018, 51, 79–91, doi:10.28934/ea.18.51.12.pp79‐91.
  • Kung, C.‐C.; Mu, J.E. Prospect of China's renewable energy development from pyrolysis and biochar applications under climate change. Renew. Sustain. Energy Rev. 2019, 114, 109343, doi:10.1016/j.rser.2019.109343.
  • Gherheș, V.; Obrad, C. Technical and Humanities Students' Perspectives on the Development and Sustainability of Artificial Intelligence (AI). Sustainability 2018, 10, 3066, doi:10.3390/su10093066.
  • Prado, R.A. La socioformación: Un enfoque de cambio educativo. Rev. Iberoam. Educ. 2018, 76, 57–82, doi:10.35362/rie7612955.
  • Martínez, J.E.; Tobón, S.; López, E. Complex Thought and Quality Accreditation of Curriculum in Online Higher Education. Adv. Sci. Lett. 2019, 25, 54–56, doi:10.1166/asl.2019.13185.
  • Universidad Tecnológica Indoamérica Modelo Educativo; Universidad Indoamérica: Quito, Ecuador, 2019. Available online: https://issuu.com/cife/docs/modelo_educativo_uti (accessed on 8 May 2021).
  • CIFE. Modelo Educativo Socioformaivo; CIFE: Morelos, Mexico, 2017. Available online: www.cife.edu.mx (accessed on 8 May 2021).
  • Universidad Nacional Hermilio Valdizan. Modelo Educativo; Unheval: Huánuco, Peru, 2017.
  • UNFV. Modelo Educativo de la UNFV, Socioformativo‐Humanista; UNFV: Lima, Peru, 2017. Available online: https://is‐ suu.com/cife/docs/modelo_educativo_unfv (accessed on 8 May 2021).
  • Serrano, R.; Macias, W.; Rodriguez, K.; Amor, M.I. Validating a scale for measuring teachers' expectations about generic com‐ petences in higher education. J. Appl. Res. High. Educ. 2019, 11, 439–451, doi:10.1108/jarhe‐09‐2018‐0192.
  • Burgos, J.A.B.; Salvador, M.R.A.; Narváez, H.O.P. Del pensamiento complejo al pensamiento computacional: Retos para la edu‐ cación contemporánea. Sophía 2016, 2, 143, doi:10.17163/soph.n21.2016.06.
  • Ortega‐Carbajal, M.F.; Hernández‐Mosqueda, J.S.; Tobón‐Tobón, S. Análisis documental de la gestión del conoci‐miento me‐ diante la cartografía conceptual. Ra Ximhai 2015, 11, 141–160.
  • Kember, D.; Leung, D.Y.P. Development of a questionnaire for assessing students' perceptions of the teaching and learning environment and its use in quality assurance. Learn. Environ. Res. 2009, 12, 15–29, doi:10.1007/s10984‐008‐9050‐7.
  • Gargallo, B.; Suárez‐Rodríguez, J.M.; Almerich, G.; Verde, I.; Iranzo, M.I.; Àngels, C. Validación del cuestionario SEQ en pobla‐ ción universitaria española. Capacidades del alumno y entorno de enseñanza/aprendizaje. Anales Psicol. 2018, 34, 519–530, doi:10.6018/analesps.34.3.299041.
  • Watson, G. Watson‐Glaser Critical Thinking Appraisal; Psychological Corporation: San Antonio, TX, USA, 1980. Available online: https://cutt.ly/3tWodUY (accessed on 8 May 2021).
  • D'Alessio, F.A.; Avolio, B.E.; Charles, V. Studying the impact of critical thinking on the academic performance of executive MBA students. Think. Ski. Creat. 2019, 31, 275–283, doi:10.1016/j.tsc.2019.02.002
  • Facione, P.; and Facione, N. The California Critical Thinking Dispositions Inventory (CCTDI): And the CCTDI Test Manual; The California Academic Press: Milbrae, CA, USA, 1992.
  • Bayram, D.; Kurt, G.; Atay, D. The Implementation of WebQuest‐supported Critical Thinking Instruction in Pre‐service English Teacher Education: The Turkish Context. Particip. Educ. Res. 2019, 6, 144–157, doi:10.17275/per.19.18.6.2.
  • Insight Assessment. California Critical Thinking Skills Test: CCTST Test Manual: "The Gold Standard" Test of Critical Thinking; Cal‐ ifornia Academic Press: San Jose, CA, USA, 2013.
  • Heilat, M.Q.; Seifert, T. Mental motivation, intrinsic motivation and their relationship with emotional support sources among gifted and non‐gifted Jordanian adolescents. Cogent Psychol. 2019, 6, 1587131, doi:10.1080/23311908.2019.1587131.
  • Kaufmann, G.; Martinsen, Ø. The AE Scale. Revised; University of Bergen, Department of General Psychology: Bergen, Norway, 1992.
  • Toapanta‐Pinta, P.; Rosero‐Quintana, M.; Salinas‐Salinas, M.; Cruz‐Cevallos, M.; Vasco‐Morales, S. Percepción de los estudian‐ tes sobre el proyecto integrador de saberes: Análisis métricos versus ordinales. Educ. Médica 2019, doi:10.1016/j.edumed.2019.10.015
  • Leach, S.; Immekus, J.C.; French, B.F.; Hand, B. The factorial validity of the Cornell Critical Thinking Tests: A multi‐analytic approach. Think. Ski. Creat. 2020, 37, 100676, doi:10.1016/j.tsc.2020.100676.
  • Amrina, Z.; Desfitri, R.; Zuzano, F.; Wahyuni, Y.; Hidayat, H.; Alfino, J. Developing Instruments to Measure Students' Logical, Critical, and Creative Thinking Competences for Bung Hatta University Students. Int. J. Eng. Technol. 2018, 7, 128–131, doi:10.14419/ijet.v7i4.9.20633
  • Poondej, C.; Lerdpornkulrat, T. The reliability and construct validity of the critical thinking disposition scale. J. Psychol. Educ. Res. 2015, 23, 23–36.
  • Said‐Metwaly, S.; Kyndt, E.; Noortgate, W.V.D. The factor structure of the Verbal Torrance Test of Creative Thinking in an Arabic context: Classical test theory and multidimensional item response theory analyses. Think. Ski. Creat. 2020, 35, 100609, doi:10.1016/j.tsc.2019.100609.
  • Burin, D.I.; Gonzalez, F.M.; Barreyro, J.P.; Injoque‐Ricle, I. Metacognitive regulation contributes to digital text comprehension in E‐learning. Metacogn. Learn. 2020, 15, 391–410, doi:10.1007/s11409‐020‐09226‐8
  • Gok, T. Development of problem solving strategy steps scale: Study of validation and reliability. Asia‐Pac. Educ. Res. 2011, 20, 151–161.
  • Penfield, R.D.; Giacobbi, J.P.R. Applying a Score Confidence Interval to Aiken's Item Content‐Relevance Index. Meas. Phys. Educ. Exerc. Sci. 2004, 8, 213–225, doi:10.1207/s15327841mpee0804_3
  • Cronbach, L.J. Coefficient alpha and the internal structure of tests. Psychometrika 1951, 16, 297–334, doi:10.1007/bf02310555
  • Raykov, T. Estimation of Composite Reliability for Congeneric Measures. Appl. Psychol. Meas. 1997, 21, 173–184, doi:10.1177/01466216970212006.
  • Hair, J.F.; Black, W.; Babin, B.; Anderson, R. Multivariate Data Analysis, 7th ed.; Pearson: London, UK, 2014. Available online: https://files.pearsoned.de/inf/ext/9781292035116 (accessed on 8 May 2021).
  • Fornell, C.; Larcker, D.F. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. J. Mark. Res. 1981, 18, 39, doi:10.2307/3151312
  • Borgonovi, F.; Greiff, S. Societal level gender inequalities amplify gender gaps in problem solving more than in academic disci‐ plines. Intelligence 2020, 79, 101422, doi:10.1016/j.intell.2019.101422.
  • Fitriani, H.; Asy'Ari, M.; Zubaidah, S.; Mahanal, S. Exploring the Prospective Teachers' Critical Thinking and Critical Analysis Skills. J. Pendidik. IPA Indones. 2019, 8, 379–390, doi:10.15294/jpii.v8i3.19434
  • Flavell, J.H. Metacognition and cognitive monitoring: A new area of cognitive‐developmental inquiry. Am. Psychol. 1979, 34, 906–911, doi:10.1037/0003‐066x.34.10.906
  • Velkovski, V. Application of the systemic analysis for solving the problems of dependent events in agricultural lands‐charac‐ teristics and methodology. Trakia J. Sci. 2019, 17, 319–323, doi:10.15547/tjs.2019.s.01.052
  • Curran, P.J.; West, S.G.; Finch, J.F. The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychol. Methods 1996, 1, 16–29, doi:10.1037/1082‐989x.1.1.16.
  • Fabrigar, L.R.; Wegener, D.T.; Maccallum, R.C.; Strahan, E.J. Evaluating the use of exploratory factor analysis in psychological research. Psychol. Methods 1999, 4, 272–299, doi:10.1037/1082‐989x.4.3.272.
  • Hu, L.T.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alterna‐ tives. Struct. Equ. Model. 1999, 6, 1–55.
  • Kaplan, D. Evaluating and Modifying Covariance Structure Models: A Review and Recommendation. Multivar. Behav. Res. 1990, 25, 137–155, doi:10.1207/s15327906mbr2502_1.
  • Putnick, D.L.; Bornstein, M.H. Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Dev. Rev. 2016, 41, 71–90, doi:10.1016/j.dr.2016.06.004.
  • Cheung, G.W.; Rensvold, R.B. Evaluating Goodness‐of‐Fit Indexes for Testing Measurement Invariance. Struct. Equ. Model. A Multidiscip. J. 2002, 9, 233–255, doi:10.1207/s15328007sem0902_5.
  • Chen, F.F. Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance. Struct. Equ. Model. A Multidiscip. J. 2007, 14, 464–504, doi:10.1080/10705510701301834.
  • Wu, H.; Leung, S.‐O. Can Likert Scales be Treated as Interval Scales?—A Simulation Study. J. Soc. Serv. Res. 2017, 43, 527–532, doi:10.1080/01488376.2017.1329775.
  • Hair, J.F.; Black, W.C.; Babin, B.J.; Anderson, R.E. Multivariate Data Analysis: A Global Perspective; Prentice Hall: Upper Saddle River, NJ, USA, 2010.
  • Luna‐Nemecio, J. Determinaciones socioambientales del COVID‐19 y vulnerabilidad económica, espacial y sanitario‐institucio‐ nal. Rev. Cienc. Soc. 2020, 26, 21–26, doi:10.31876/rcs.v26i2.32419
  • Roozenbeek, J.; Schneider, C.R.; Dryhurst, S.; Kerr, J.; Freeman, A.L.J.; Recchia, G.; van der Bles, A.M.; van der Linden, S. Sus‐ ceptibility to misinformation about COVID‐19 around the world. R. Soc. Open Sci. 2020, 7, 1–15, doi:10.1098/rsos.201199
  • Lawrence, R.J. Responding to COVID‐19: What's the Problem? J. Hered. 2020, 97, 583–587, doi:10.1007/s11524‐020‐00456‐4.
  • Connelly, L.M. What Is Factor Analysis? Medsurg Nurs. 2019, 28, 322–330.
  • Disabato, D.J.; Goodman, F.R.; Kashdan, T.B.; Short, J.L.; Jarden, A. Different types of well‐being? A cross‐cultural examination of hedonic and eudaimonic well‐being. Psychol. Assess. 2016, 28, 471–482, doi:10.1037/pas0000209.
  • Gau, J.M. The Convergent and Discriminant Validity of Procedural Justice and Police Legitimacy: An Empirical Test of Core Theoretical Propositions. J. Crim. Justice 2011, 39, 489–498, doi:10.1016/j.jcrimjus.2011.09.004.
  • Cseh, M.; Crocco, O.S.; Safarli, C. Teaching for Globalization: Implications for Knowledge Management in Organizations. In Social Knowledge Management in Action; Springer: Berlin/Heidelberg, Germany, 2019; pp. 105–118
  • Knutson, J.S.; Friedl, A.S.; Hansen, K.M.; Hisel, T.Z.; Harley, M.Y. Convergent Validity and Responsiveness of the SULCS. Arch. Phys. Med. Rehabil. 2019, 100, 140–143.e1, doi:10.1016/j.apmr.2018.07.433.
  • Vaughn, M.G.; Roberts, G.; Fall, A.‐M.; Kremer, K.; Martinez, L. Preliminary validation of the dropout risk inventory for middle and high school students. Child. Youth Serv. Rev. 2020, 111, 104855, doi:10.1016/j.childyouth.2020.104855
  • Degener, S.; Berne, J. Complex Questions Promote Complex Thinking. Read. Teach. 2016, 70, 595–599, doi:10.1002/trtr.1535
  • Viguri, M. Science of complexity vs. complex thinking keys for a critical reading of the concept of scientific in Carlos Reynoso. Pensam. Rev. Investig. Inf. Filos. 2019, 75, 87–106, doi:10.1016/j.tsc.2021.100794.
  • Mara, J.; Lavandero, J.; Hernández, L.M. The educational model as the foundation of university action: The experience of the Technical University of Manabi, Ecuador. Rev. Cuba. Educ. Super. 2018, 37, 151–164.