The art of measuring art: Methodical development and validation of RizbA, a transdisciplinary rating instrument for two-dimensional pictorial works

Although psychological research on art has some tradition, measuring artworks and their formal content has received surprisingly little attention in quantitative research. One reason for this neglect may be the humanistic or qualitative orientation of art sciences; another may be the lack of reliable, valid, and objective instruments for measuring pictorial expression. By bridging art theory and psychometrics, the Rating instrument for two-dimensional pictorial works (RizbA) addresses this gap.

The scale was developed and validated in four validation studies on a total of 899 pictorial works by contemporary artists and nonprofessionals, being rated by a total of 1,577 art experts. A 26-item version was developed in the first study. The second study validated the scale on pictorial works by nonprofessionals, the third on pictures by contemporary artists. Statistical quality criteria such as item difficulty, capacity of differentiation between images, test-retest reliability, inter-rater reliability, Principal Component Analysis and Tucker's coefficients of congruence were computed. The fourth study specified three path models and conducted a Confirmatory Factor Analysis (CFA). Three further methodical studies developed a machine learning approach, validated a version usable for non-art experts, and provided an item pool for three-dimensional works. Three application studies used the scale on image samples by persons with chronic pain, depression, delirium, and by children.

The results suggest high capacity of differentiation (𝜂_p² [.28, .90]), high test-retest reliability (r [.86, .92]), and modest to excellent inter-rater reliability (ICC [.53, .92]). Regarding the CFA, only one model partially suggests an acceptable fit.

The findings imply reliability and generalizability while the question of the factor structure persists and speaks to a methodological gap between empirical evidence and theory. Since art is ambiguous, with various approaches to its analysis, postdisciplinary approaches are needed to do justice to it.