A Taxonomy classifying UI components in desktop interfaces

Taxonomy

This publication presents a structured taxonomy for desktop User Interface (UI) Components. The development methodology foundation is extensively detailed in [1]. This taxonomy categorizes UI components into hierarchical levels of abstraction, as described in [2]: System, Application, UI Group, and UI Element.

The taxonomy is presented in a table format, organized into the following columns:

• UI Component Type: Identifies the specific UI Component being described, with examples including Icons, Taskbars, etc.
• Level: Specifies the component's hierarchical position by identifying the level of abstraction to which it belongs.
• Detected by: Lists the model employed to detect a particular type of UI Component, based on a multi-modal approach for detecting UI Hierarchies from desktop screenshots, as discussed in [1], where M1, M2, M3, M4 and M5 are described.
• Description: Provides an explanation of the criteria for classifying a specific UI Component type.

This taxonomy facilitates a deeper understanding and navigation of the complexities inherent in desktop UIs.

References

    1. A. Martínez-Rojas, A. Rodríguez-Ruíz, J.G. Enríquez, and A. Jiménez-Ramírez (2024, September). What’s Behind the Screen? Unveiling UI Hierarchies in Process-Related UI logs. In International Conference on Business Process Management. Cham: Springer International Publishing.

    2. Abb, L., Rehse, J.R.: A reference data model for process-related user interaction logs. In: International
Conference on Business Process Management. pp. 57–74. Springer (2022)