Measuring library consortia performance

This conference paper was presented at the 14th International Conference on Performance Measurement in Libraries, 2-4 November 2021.

A services’ performance measurement should be a major concern of any library consortium because it is a useful tool in the ever antagonizing world of limited library resources.

ISO 11620 indicators measure the performance of a single library. The question that arises is whether this standard could assess the performance of a consortium or the services a consortium provides. In this process, do we need to adjust some of the performance indicators of the ISO11620 standard?

Design, methodology, or approach: The investigation of the research question is analyzed to the following issues:

1. Determine which services consortia are most likely to offer to the participating libraries.
2. Investigate whether ISO 11620 provides the appropriate indicators to measure the performance of the consortia services.
3. Propose modifications –if needed- in the definitions or the data collection methodology for some of the performance indicators.
4. Examine whether some of the performance indicators might be crucial (key indicators) for the assessment of library consortia.

Findings & limitations (as applicable):

Issue 1: The most common services were: Cooperative Collection Development (including cooperative purchasing, electronic content licensing, and shared print storage), Digitization Services, Institutional Repositories, Programs for Users, Resource Sharing & ILL, Shared ILS & Collaborative Cataloguing, Staff Training & Consulting, and Technology management services.

Issue 2: For each service, we found at least one suitable performance indicator (p.i) that could be applied. For instance, the p.i. ‘Acquisition Cost per Collection Use’ could be applied to the service Cooperative Collection Development to assess its cost-efficiency. In this case, the consortium’s expenditure and instances of collection use for all library members should be counted. The indicators ‘Speed of Interlibrary Lending’ and ‘Percentage of Successful Interlibrary Loans’ could be calculated to assess the interlibrary loan for physical documents and/or the resource sharing for electronic document delivery service.

The standard also describes p.i. for assessing the consortium’s management, its general impact, and ability to cover the needs of the audience. For example, “Percentage of Required Titles in the Collection” is used to assess the fit of the collection to the users’ requirements. To apply this p.i., the required titles should be owned or licensed by the consortium or loaned between the participating libraries.

Issue 3: Certain p.i. could be adjusted to measure performance.

Issue 4: Several p.i. could be considered key, although this is heavily dependent on the purpose of the evaluation and the application profile.

Limitations: this model could be confirmed with real-world data; data from different consortia need to be collected with the same methodology to serve comparison purposes.

Conclusions and how findings have been applied:

We found that ISO 11620 p.i. can affirmatively measure the most common services consortia offer to their members. Applying an ISO standard has certain benefits one could reap: easy to apply, world-recognized, an established methodology to make comparisons, a well-known tool that came out of global research and consideration, to name a few.

The proposed approach could contribute to the discussion on the performance measurement for consortia services and offer assistance in implementing the ISO11620 as a tool that libraries are already familiar with.