Building trust through values: Measurement of Community Health Indicators (MoCHI) project report

From January through December 2025, funded by the Digital Infrastructure Insights Fund (D//F), IOI conducted the MoCHI project to ask: How applicable and useful are evaluation frameworks and metrics in incentivizing investment in and adoption of open infrastructure for research? Using interviews, co-design workshops, and participatory validation with infrastructure providers, funders, and adopters, the project tested the hypothesis that identifying shared values and measuring adherence to them could help strengthen ecosystem resilience. Our central finding was more nuanced: stakeholders operate in constant tension between aspirational values (open principles, community service, academy-owned outputs) and practical constraints (budgets, procurement processes, institutional requirements). Successful infrastructure evaluation requires tools that help stakeholders navigate these tensions.

Through interviews and workshops, we identified 11 themes that stakeholders mentioned as key for their evaluation process: 

• Data Ownership, Portability, and Control: the only theme rated "Very Important" by all stakeholder groups. Stakeholders care deeply about maintaining control over what they put into systems.

• Affordability: About value relative to cost, not simply low price. Contextual: a threshold requirement for some; a value-for-investment calculus for others.

• Technical Requirements: Features, integrations, and compatibility with existing skills. Should include ease of adoption, not just a feature checklist.

• Transparent Governance: Clear understanding of who makes decisions and how. 

• Fiscal Security: Adequate, sustainable funding to cover costs and build reserves. Critical but hardest to assess; negative indicators are easier to spot than positive proof.

• Support and Technical Training: Broader than official documentation — includes peer support, third-party resources, vendor ecosystems, and self-service capacity.

• Policy and Regulatory Compliance: Binary threshold: non-negotiable when it applies; irrelevant when it doesn’t.

• Sense of Community and Belonging: Effective community input mechanisms and inclusive culture. Distinction between feeling heard and having actual decision-making power.

• Longevity and Embeddedness: Well-established, actively used, known in the field. Can signal stability or technical debt; past resilience does not guarantee future resilience.

• Usage and Adoption:  Relevant to and adopted by peer organizations. Innovation vs. stability tension: high adoption signals quality but can preclude transformational infrastructure.

• Values Alignment and Community Orientation: Captures whether infrastructure is mission-driven vs. profit-driven and genuinely puts community first.

In addition, the concept of resilience (financial, technical, political, and organizational) emerged repeatedly across workshops and themes as a critical forward-looking dimension not fully captured by any single theme.

Values shape behaviour, but don’t determine it. Stakeholders make pragmatic decisions constrained by cost, capacity, institutional policies, and switching costs, often accepting trade-offs that don’t fully reflect their stated ideals. The relative importance of each theme is strongly related to context, including the needs of a particular stakeholder and their community and the timing of an evaluation exercise.  

The way our participants talked about open infrastructure often, ultimately, came down to trust: that scholarly outputs and research data will remain accessible, that infrastructure will serve community needs over pursuit of profit, that decisions will be made transparently, and that participants can exit if things change. The evaluation themes we identified are some of the criteria that different stakeholders called on in developing and articulating that trust.