Building on scientific literacy in evolution towards scientifically responsible Europeans, COST Action CA17127
Published March 9, 2023 | Version v1
Project deliverable Open

Assessing in-person and online informal learning activities to promote understanding of evolution - a guide for practitioners

Authors/Creators

  • 1. Evolutionary Knowledge for Everyone (EvoKE), Paris, France
  • 2. Center for the promotion of science, Belgrade, Serbia
  • 3. Department of Evolutionary Biology, Institute for Biological Research "Siniša Stanković"—National Institute of the Republic of Serbia, University of Belgrade, Belgrade, Serbia
  • 4. University of Western Macedonia, Florina, Greece
  • 5. Rhizome Association, Paris, France

Description

Informal learning activities are a powerful medium to promote understanding of evolution, as they have a wide reach. Although the diversity of the types of informal learning activities is large, a common issue is that learning outcomes are seldom assessed, and rarely a part of the activity design. In this report, we present the ethical considerations and theoretical perspective that can help practitioners adopt assessment as an integral element of the activity design. We also describe an exploratory study designing assessments to assess knowledge, engagement, motivation, utility and attitudes for two contemporary examples of in-person and online learning activities. We conclude by reflecting on our process and providing recommendations to encourage practitioners to improve their practice by incorporating assessments in their activities.

Files

Assessing in-person and online informal learning activities to promote understanding of evolution - a guide for practitioners (1).pdf

Files (3.2 MB)

Additional details

References

  • Adnađević, T., Milosevic, T., & Radovčić, D. (2020). Exploratory study of evolution-themed, non-formal education in Europe. Zenodo. https://doi.org/10.5281/zenodo.3712725
  • All European Academies [ALLEA]. (2017). The European Code of Conduct for Research Integrity—ALLEA. https://allea.org/code-of-conduct/
  • Allen, S. (2002). Looking for learning in visitor talk: A methodological exploration. In G. Leinhardt, K. Crowley, & K. Knutson (Eds.), Learning Conversations in Museums (pp. 259–303). Lawrence Erlbaum Associates. https://www.exploratorium.edu/files/vre/pdf/Allen_chapter_sentweb2.pdf
  • Allen, S. (2004). Designs for learning: Studying science museum exhibits that do more than entertain. Science Education, 88(S1), S17–S33. https://doi.org/10.1002/sce.20016
  • Anderson, D. L., Fisher, K. M., & Norman, G. J. (2002). Development and evaluation of the conceptual inventory of natural selection. Journal of Research in Science Teaching, 39(10), 952–978. https://doi.org/10.1002/tea.10053
  • Archer, L., Godec, S., Patel, U., Dawson, E., & Calabrese Barton, A. (2022). 'It really has made me think': Exploring how informal STEM learning practitioners developed critical reflective practice for social justice using the Equity Compass tool. Pedagogy, Culture & Society, 0(0), 1–23. https://doi.org/10.1080/14681366.2022.2159504
  • Banks, J. A., Au, K. H., Ball, A. F., Bell, P., Gordon, E. W., Gutiérrez, K. D., Heath, S. B., Lee, C. D., Lee, Y., Mahiri, J., Nasir, N. S., Valdés, G., & Zhou, M. (2007). Learning in and out of school in diverse environments: Life-Long, Life-Wide, Life-Deep. The LIFE Center (University of Washington, Stanford University and SRI) & the Center for Multicultural Education, University of Washington. https://education.uw.edu/sites/default/files/cme/docs/LEARNING%20LIFE%20REPORT.pdf
  • Barone, L., & Buntin, M. (2014). Human evolution acceptance among visitors to the Milwaukee Public Museum. Informal Learning Review, 124, 9–13.
  • Beniermann, A., Kuschmierz, P., Pinxten, R., Aivelo, T., Bohlin, G., Brennecke, J. S., Cebesoy, U. B., Cvetković, D., Đorđević, M., Dvořáková, R. M., Futo, M., Nicoleta, G., Korfiatis, K., Lendvai, A., Mogias, A., Paolucci, S., Petersson, M., Pietrzak, B., Porozovs, J., … Graf, D. (2021). Evolution Education Questionnaire on Acceptance and Knowledge (EEQ)—Standardised and ready-to-use protocols to measure acceptance of evolution and knowledge about evolution in an international context. Zenodo. https://doi.org/10.5281/zenodo.5798227
  • Bishop, B. A., & Anderson, C. W. (1990). Student conceptions of natural selection and its role in evolution. Journal of Research in Science Teaching, 27(5), 415–427. https://doi.org/10.1002/tea.3660270503
  • Borun, M., Massey, C., & Lutter, T. (1993). Naive Knowledge and the Design of Science Museum Exhibits. Curator: The Museum Journal, 36(3), 201–219. https://doi.org/10.1111/j.2151-6952.1993.tb00794.x
  • Bray Speth, E., Long, T. M., Pennock, R. T., & Ebert-May, D. (2009). Using Avida-ED for Teaching and Learning About Evolution in Undergraduate Introductory Biology Courses. Evolution: Education and Outreach, 2(3), 415–428. https://doi.org/10.1007/s12052-009-0154-z
  • Catley, K. M., & Novick, L. R. (2009). Digging deep: Exploring college students' knowledge of macroevolutionary time. Journal of Research in Science Teaching, 46(3), 311–332. https://doi.org/10.1002/tea.20273
  • Chen, S. (2006). Development of an instrument to assess views on nature of science and attitudes toward teaching science. Science Education, 90(5), 803–819. https://doi.org/10.1002/sce.20147
  • Clayton, S., Fraser, J., & Saunders, C. D. (2009). Zoo experiences: Conversations, connections, and concern for animals. Zoo Biology, 28(5), 377–397. https://doi.org/10.1002/zoo.20186
  • Csikszentmihalyi, M., & Hermanson, K. (1995). Intrinsic motivation in museums: Why does one want to learn? In J. H. Falk & L. D. Dierking (Eds.), Public institutions for personal learning: Establishing a research agenda. American Association of Museums. https://arts.berkeley.edu/wp-content/uploads/2016/01/Csikszentmihalyi-Hermanson-1995_Intrinsic-Motivation-in-Museums.pdf
  • de Lima, J. (2021). Contextual Influences on Undergraduate Biology Students' Reasoning and Representations of Evolutionary Concepts. Michigan State University.
  • de Lima, J., Adnađević, T., Barstad, J., Georgiou, M., Holm, S., Kenig, B., Martinelli, L., Nogueira, T., Savković, U., Tsopoglou-Gkina, D., & Milosevic, T. (2023). Developing non-formal learning activities focused on increasing evolutionary knowledge and scientific literacy. https://doi.org/10.5281/zenodo.7711038
  • Diamond, J., Horn, M., & Uttal, D. H. (2016). Practical evaluation guide: Tools for museums and other informal educational settings (3rd Edition). Rowman & Littlefield.
  • Dierking, L. D., Adelman, L. M., Ogden, J., Lehnhardt, K., Miller, L., & Mellen, J. D. (2004). Using a Behavior Change Model to Document the Impact of Visits to Disney's Animal Kingdom: A Study Investigating Intended Conservation Action. Curator: The Museum Journal, 47(3), 322–343. https://doi.org/10.1111/j.2151-6952.2004.tb00128.x
  • Dufty, N. (2020). Disaster Education, Communication and Engagement. John Wiley & Sons.
  • Dunk, R. D. P., Petto, A. J., Wiles, J. R., & Campbell, B. C. (2017). A multifactorial analysis of acceptance of evolution. Evolution: Education and Outreach, 10(1), 4. https://doi.org/10.1186/s12052-017-0068-0
  • Fahr, R. (2005). Loafing or learning? —The demand for informal education. European Economic Review, 49(1), 75–98. https://doi.org/10.1016/j.euroecorev.2004.03.002
  • Falk, J. H., & Adelman, L. M. (2003). Investigating the impact of prior knowledge and interest on aquarium visitor learning. Journal of Research in Science Teaching, 40(2), 163–176. https://doi.org/10.1002/tea.10070
  • Falk, J. H., Reinhard, E., Vernon, C., Bronnenkant, K., Heimlich, J., & Deans, N. (2007). Why Zoos & Aquariums Matter: Assessing the Impact of a Visit to a Zoo or Aquarium. Association of Zoos & Aquariums. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.574.3479&rep=rep1&type=pdf
  • Falk, J. H., & Storksdieck, M. (2005). Using the contextual model of learning to understand visitor learning from a science center exhibition. Science Education, 89(5), 744–778. https://doi.org/10.1002/sce.20078
  • Falk, J. H., Storksdieck, M., & Dierking, L. D. (2007). Investigating public science interest and understanding: Evidence for the importance of free-choice learning. Public Understanding of Science, 16(4), 455–469. https://doi.org/10.1177/0963662506064240
  • Feng, S., Hossain, L., & Paton, D. (2017). Harnessing informal education for community resilience. Disaster Prevention and Management, 27(1), 43–59. https://doi.org/10.1108/DPM-07-2017-0157
  • Frierson, H. T., Hood, S., & Hughes, G. B. (2002). A guide to conducting culturally responsive evaluations. In J. Frechtling, The 2002 User-Friendly Handbook for Project Evaluation. The National Science Foundation. https://www.nsf.gov/pubs/2002/nsf02057/nsf02057.pdf
  • Furrow, R. E., & Hsu, J. L. (2019). Concept inventories as a resource for teaching evolution. Evolution: Education and Outreach, 12(1), 2. https://doi.org/10.1186/s12052-018-0092-8
  • Garvin-Doxas, K., & Klymkowsky, M. W. (2008). Understanding Randomness and its Impact on Student Learning: Lessons Learned from Building the Biology Concept Inventory (BCI). CBE—Life Sciences Education, 7(2), 227–233. https://doi.org/10.1187/cbe.07-08-0063
  • Godinez, A. M., & Fernandez, E. J. (2019). What Is the Zoo Experience? How Zoos Impact a Visitor's Behaviors, Perceptions, and Conservation Efforts. Frontiers in Psychology, 10. https://www.frontiersin.org/articles/10.3389/fpsyg.2019.01746
  • Groß, J., Kremer, K., & Arnold, J. (2019). Evolution Learning and Creationism: Thinking in Informal Learning Environments. In U. Harms & M. J. Reiss (Eds.), Evolution Education Re-considered. Springer, Cham. https://doi.org/10.1007/978-3-030-14698-6
  • Guichard, J. (1995). Designing tools to develop the conception of learners. International Journal of Science Education, 17(2), 243–253. https://doi.org/10.1080/0950069950170208
  • Guzey, S. S., Harwell, M., & Moore, T. (2014). Development of an Instrument to Assess Attitudes Toward Science, Technology, Engineering, and Mathematics (STEM). School Science and Mathematics, 114(6), 271–279. https://doi.org/10.1111/ssm.12077
  • Ha, M., Haury, D. L., & Nehm, R. H. (2012). Feeling of certainty: Uncovering a missing link between knowledge and acceptance of evolution. Journal of Research in Science Teaching, 49(1), 95–121. https://doi.org/10.1002/tea.20449
  • Heddy, B. C., & Nadelson, L. S. (2012). A Global Perspective of the Variables Associated with Acceptance of Evolution. Evolution: Education and Outreach, 5(3), 412–418. https://doi.org/10.1007/s12052-012-0423-0
  • Huba, M. E., & Freed, J. E. (2000). Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning. Allyn & Bacon.
  • Hughes, R. M., Nzekwe, B., & Molyneaux, K. J. (2013). The Single Sex Debate for Girls in Science: A Comparison Between Two Informal Science Programs on Middle School Students' STEM Identity Formation. Research in Science Education, 43(5), 1979–2007. https://doi.org/10.1007/s11165-012-9345-7
  • Humphrey, T., Gutwill, J., & Exploratorium APE Team. (2017). Fostering active prolonged engagement. Taylor & Francis.
  • Huneman, P., & Walsh, D. M. (Eds.). (2017). Challenging the modern synthesis: Adaptation, development, and inheritance. Oxford University Press.
  • Kalinowski, S. T., Leonard, M. J., & Taper, M. L. (2016). Development and Validation of the Conceptual Assessment of Natural Selection (CANS). CBE—Life Sciences Education, 15(4), ar64. https://doi.org/10.1187/cbe.15-06-0134
  • Klymkowsky, M. W., Underwood, S. M., & Garvin-Doxas, R. K. (2010). Biological Concepts Instrument (BCI): A diagnostic tool for revealing student thinking (arXiv:1012.4501). arXiv. https://doi.org/10.48550/arXiv.1012.4501
  • Korn, R. (2003). Summative Evaluation of Vanishing Wildlife. Monterey Bay Aquarium. https://www.informalscience.org/sites/default/files/report_45.pdf
  • Kuschmierz, P., Beniermann, A., & Graf, D. (2020). Development and evaluation of the knowledge about evolution 2.0 instrument (KAEVO 2.0). International Journal of Science Education, 42(15), 2601–2629. https://doi.org/10.1080/09500693.2020.1822561
  • Liang, L. L., Chen, S., Chen, X., Kaya, O. N., Adams, A. D., Macklin, M., & Ebenezer, J. (2006). Student Understanding of Science and Scientific Inquiry (SUSSI): Revision and Further Validation of an Assessment Instrument. Annual Conference of the National Association for Research in Science Teaching (NARST), San Francisco, CA.
  • Lovelace, M., & Brickman, P. (2013). Best Practices for Measuring Students' Attitudes toward Learning Science. CBE—Life Sciences Education, 12(4), 606–617. https://doi.org/10.1187/cbe.12-11-0197
  • Martin, L. M. W. (2004). An emerging research framework for studying informal learning and schools. Science Education, 88(S1), S71–S82. https://doi.org/10.1002/sce.20020
  • McNamara, P. (2005). Amazing Feats of Aging Exhibit: A Summative Evaluation Report. Oregon Museum of Science and Industry. https://omsi.edu/sites/default/files/afoasum.pdf
  • Meisner, R., vom Lehn, D., Heath, C., Burch, A., Gammon, B., & Reisman, M. (2007). Exhibiting Performance: Co‐participation in science centres and museums. International Journal of Science Education, 29(12), 1531–1555. https://doi.org/10.1080/09500690701494050
  • Moharreri, K., Ha, M., & Nehm, R. H. (2014). EvoGrader: An online formative assessment tool for automatically evaluating written evolutionary explanations. Evolution: Education and Outreach, 7(15), 1–14. https://doi.org/10.1186/s12052-014-0015-2
  • Moore, R. W., & Foy, R. L. H. (1997). The scientific attitude inventory: A revision (SAI II). Journal of Research in Science Teaching, 34(4), 327–336. https://doi.org/10.1002/(SICI)1098-2736(199704)34:4<327::AID-TEA3>3.0.CO;2-T
  • Myers Jr., O. E., Saunders, C. D., & Birjulin, A. A. (2004). Emotional Dimensions of Watching Zoo Animals: An Experience Sampling Study Building on Insights from Psychology. Curator: The Museum Journal, 47(3), 299–321. https://doi.org/10.1111/j.2151-6952.2004.tb00127.x
  • Nadelson, L. S., & Southerland, S. (2012). A More Fine-Grained Measure of Students' Acceptance of Evolution: Development of the Inventory of Student Evolution Acceptance—I-SEA. International Journal of Science Education, 34(11), 1637–1666. https://doi.org/10.1080/09500693.2012.702235
  • National Research Council [NRC]. (2000). How People Learn: Brain, Mind, Experience, and School: Expanded Edition. The National Academies Press. https://doi.org/10.17226/9853
  • National Research Council [NRC]. (2001a). Knowing What Students Know: The Science and Design of Educational Assessment. The National Academies Press. https://doi.org/10.17226/10019.
  • National Research Council [NRC]. (2001b). The Nature of Assessment and Reasoning from Evidence. In Knowing What Students Know: The Science and Design of Educational Assessment. The National Academies Press. https://doi.org/10.17226/10019
  • National Research Council [NRC]. (2007). Taking Science to School: Learning and Teaching Science in Grades K-8. The National Academies Press. https://doi.org/10.17226/11625
  • National Research Council [NRC]. (2009). Learning Science in Informal Environments: People, Places, and Pursuits. The National Academies Press. https://doi.org/10.17226/12190
  • Nehm, R. H., Beggrow, E. P., Opfer, J. E., & Ha, M. (2012). Reasoning About Natural Selection: Diagnosing Contextual Competency Using the ACORNS Instrument. The American Biology Teacher, 74(2), 92–98. https://doi.org/10.1525/abt.2012.74.2.6
  • Nehm, R. H., & Ha, M. (2011). Item feature effects in evolution assessment. Journal of Research in Science Teaching, 48(3), 237–256. https://doi.org/10.1002/tea.20400
  • Nehm, R. H., & Reilly, L. (2007). Biology Majors' Knowledge and Misconceptions of Natural Selection. BioScience, 57(3), 263–272. https://doi.org/10.1641/B570311
  • Nehm, R. H., & Schonfeld, I. S. (2008). Measuring knowledge of natural selection: A comparison of the CINS, an open-response instrument, and an oral interview. Journal of Research in Science Teaching, 45(10), 1131–1160. https://doi.org/10.1002/tea.20251
  • Opfer, J. E., Nehm, R. H., & Ha, M. (2012). Cognitive Foundations for Science Assessment Design: Knowing What Students Know About Evolution. Journal of Research in Science Teaching, 49(6), 744–777. https://doi.org/10.1002/tea.21028
  • Papale, F., Saget, J., & Bapteste, É. (2020). Networks Consolidate the Core Concepts of Evolution by Natural Selection. Trends in Microbiology, 28(4), 254–265. https://doi.org/10.1016/j.tim.2019.11.006
  • Patrick, P. G. (2017). Informal Science Educators and the Nine Dimensions of Reflective Practice. In P. G. Patrick (Ed.), Preparing Informal Science Educators: Perspectives from Science Communication and Education (pp. 41–65). Springer International Publishing. https://doi.org/10.1007/978-3-319-50398-1_3
  • Price, R. M., Andrews, T. C., McElhinny, T. L., Mead, L. S., Abraham, J. K., Thanukos, A., & Perez, K. E. (2014). The Genetic Drift Inventory: A Tool for Measuring What Advanced Undergraduates Have Mastered about Genetic Drift. CBE—Life Sciences Education, 13(1), 65–75. https://doi.org/10.1187/cbe.13-08-0159
  • Roberts, K., & Hughes, R. (2022). Recognition Matters: The Role of Informal Science Education Programs in Developing Girls' Science Identity. Journal for STEM Education Research. https://doi.org/10.1007/s41979-022-00069-3
  • Rockman, S., Bass, K., & Borse, J. (2007). Media-Based Learning Science in Informal Environments Commissioned Paper. Background paper for the Learning Science in Informal Environments Committee of the National Research Council. https://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_080086.pdf
  • Rüber, I. E., & Bol, T. (2017). Informal Learning and Labour Market Returns. Evidence from German Panel Data. European Sociological Review, 33(6), 765–778. https://doi.org/10.1093/esr/jcx075
  • Sbeglia, G. C., & Nehm, R. H. (2020). Illuminating the complexities of conflict with evolution: Validation of the scales of evolutionary conflict measure (SECM). Evolution: Education and Outreach, 13(1), 23. https://doi.org/10.1186/s12052-020-00137-5
  • Schön, D. A. (1992). The Reflective Practitioner: How Professionals Think in Action. Routledge. https://doi.org/10.4324/9781315237473
  • Schon, J. A. (2015). Science Identity in Informal Education [University of Idaho]. https://www.proquest.com/openview/a1021a7e5e79c309785aaefed45d1866/1?pq-origsite=gscholar&cbl=18750
  • Serrell, B. (1997). Paying Attention: The Duration and Allocation of Visitors' Time in Museum Exhibitions. Curator: The Museum Journal, 40(2), 108–125. https://doi.org/10.1111/j.2151-6952.1997.tb01292.x
  • Sherman, W. C., Minier, D. E., Meyers, C. N., & Myers, M. L. (2022). The More the Merrier? Zoo Visitors' Affective Responses and Perceptions of Welfare across an Increase in Giraffe Density. Journal of Zoological and Botanical Gardens, 3(2), Article 2. https://doi.org/10.3390/jzbg3020023
  • Simpkins, S. C., Little, P. M. D., & Weiss, H. (2004). Understanding and Measuring Attendance in Out-of-School Time Programs (No. 7; Issues and Opportunities in Out- of-School Time Evaluation). Harvard Family Research Project. https://archive.globalfrp.org/evaluation/publications-resources/understanding-and-measuring-attendance-in-out-of-school-time-programs
  • Smith, M. U., Snyder, S. W., & Devereaux, R. S. (2016). The GAENE—Generalized Acceptance of EvolutioN Evaluation: Development of a new measure of evolution acceptance. Journal of Research in Science Teaching, 53(9), 1289–1315. https://doi.org/10.1002/tea.21328
  • Staus, N. L., & Falk, J. H. (2017). The Role of Emotion in Informal Science Learning: Testing an Exploratory Model. Mind, Brain, and Education, 11(2), 45–53. https://doi.org/10.1111/mbe.12139
  • Stevens, R., & Martell, S. T. (2003). Leaving a Trace: Supporting museum visitor interaction and interpretation with digital media annotation systems. Journal of Museum Education, 28(2), 25–31. https://doi.org/10.1080/10598650.2003.11510479
  • Tai, R. H., Qi Liu, C., Maltese, A. V., & Fan, X. (2006). Planning Early for Careers in Science. Science, 312(5777), 1143–1144. https://doi.org/10.1126/science.1128690
  • URPP Evolution in Action: From Genomes to Ecosystems. (2021). Special exhibition: Evolution happens! University of Zurich. http://www.evolution.uzh.ch/en/evolutionhappens.html
  • WG3 EuroScitizen. (2022a, April 6). 'Evolution decoded', Public workshop for educators, science communicators and artists. http://www.euroscitizen.eu/2022/04/06/public-workshop-evolution-decoded-for-educators-science-communicators-and-artists/
  • WG3 EuroScitizen. (2022b, May 9). 'Assessment decoded: A follow-up to Evolution decoded', Public workshop for educators, science communicators and non-formal activity creators. http://www.euroscitizen.eu/2022/05/09/public-workshop-assessment-decoded/