Investigation of Performance Indicators in the Strategic Plans of Public Universities in Turkey

The purpose of this study is to develop performance indicators in line with these goals and objectives in the light of domestic and foreign literature by examining the strategic goals and objectives in the strategic plans of public universities in Turkey. Within the scope of the research, firstly strategic plans of many universities operating at home and abroad were examined and a performance indicator pool consisting of 300 criteria was created in the light of the strategic plans of 11 different universities selected by purposeful sampling. Also Reports of the Council of Higher Education (YÖK) and Ministry of Development on strategic planning were examined. In the second stage, expert opinion was received from 2 faculty members working in the field of education management, and the 300 item list was reduced to 45 performance indicators under 6 main strategic dimensions. In the third and final stage of the research, performance indicators were classified according to their importance by using AHP (Analytical Hierarchy Process) method, which is a multi-criteria decision-making technique. The AHP study was carried out with 10 academics who had scientific studies in the field of strategic planning and worked in university administration. Finally, the performance indicators created for each strategic dimension are listed according to their criterion weights.


Introduction
The strategic planning in an institution includes the participation of staff at all levels and the full support of the institution manager. In this process; expectations of stakeholders and policy makers play an active role to determine the mission, objectives and performance measurement of the organization. Strategic planning helps in answering 4 basic questions for an organization (Ministry of Development, 2018a).
• How can we reach where we want to go? • How do we measure and evaluate our success?
The answers given to these questions constitute the strategic planning process. According to the Council of Higher Education (YÖK), the purpose of the strategic planning process is to determine the mission, vision and basic values of higher education institutions and the faculties, schools, conservatories, vocational schools, institutes, research centers, administrative units (library, computing etc.) and other units to establish their goals in line with the strategies of the relevant institutions (YÖK, 2007). At the same time, creating a system to determine and monitor performance criteria for the improvement of the quality of higher education institutions is a part of this process.
The performance program refers to the description of the priority strategic goals and objectives for the implementation of a financial system in strategic planning created by institutions and organizations including appropriate performance targets, activities, sources, and performance evaluations. Public administrations, budgets and resource allocation on a program and project basis in order to provide some standards of public services; based on their strategic plans, annual goals and objectives and performance indicators (Yüksel, 2014). Performance indicators are the tools to measure the success of the strategic plan and especially the results of implementation. Performance indicators are used to measure and evaluate the results achieved in the fulfillment of the strategic goals and objectives of organizations and form the basis for performance measurements. These indicators are classified as input, output, result, effectiveness and quality indicators (Altun, 2019).

Literature Review
Types of performance indicators can be briefly explained as follows (Hastürk, 2009): Input indicators: The human financial and physical resources needed to produce a product or service. The input indicators reflect the initial state, which is the basis for measuring. Output indicators: The amount of products and services produced. Although output indicators provide information about the quantity of goods and services produced, they are not alone explanatory about whether goals and objectives are achieved or the quality of the goods or services produced and the effectiveness of the production process. Productivity indicators: Input or cost per unit of output. It shows the relationship between inputs and outputs. Outcome indicators: They show how and to what extent the outputs achieved are successful in achieving strategic goals and objectives. The level of success in achieving the targeted results is expressed by efficiency. Result indicators are the most important performance indicators in terms of revealing whether strategic goals and objectives have been achieved. Quality indicators: The level reached in meeting the expectations of those who benefit from goods or services or those concerned (measures such as reliability, accuracy, behavior, sensitivity and integrity).
Creating and evaluating performance indicators is possible only by providing appropriate data and statistics. The existence of accurate and consistent data suitable for the purpose, creation of performance indicators, measurement and evaluation of performance is an indispensable prerequisite (Mutluer, Öner & Kesik, 2005).
The characteristics that performance indicators should have are stated below as items (Yüksel, 2014): Meaningful and connected: Providing direct and meaningful information directly related to the mission, goals and objectives. Institutionally accepted: Utilization of policy and budget decisions in the institution. Balanced: It includes different indicator types in order to provide a clear picture about performance. Clear, well defined and simple: Easy to understand, easy to calculate, easy to interpret, not to misunderstand. Reliable, consistent, updated and timely: The data it relies on are accurate and credible; balance with the correctness of data and the production of data on time. Comparable: To be suitable for internal and external comparisons and for years based comparisons. Verifiable: Processes regarding data collection / production are not doubtful and clearly documented.
No adverse effects: Not promoting negative, unwanted or useless trends.
However, Kueng (2000) defined the basic features of performance indicators as follows: 1. Performance indicators should be in a measurable format indicating number, ratio or quantitative result. 2. Performance indicators should be sensitive to change. 3. A performance indicators should be linear. Linearity means that performance changes through a linear relationship with the value of the indicator. 4. A performance indicator should be reliable. Reliability means that performance must be calculated correctly, both in routine conditions and in unexpected situations. 5. Performance indicators should be efficient, clear and understandable. 6. Performance indicators should be directed towards improvement within the strategic plan. The progressive performance indicator expresses the necessary changes to ensure competitive business performance.
According to Parmenter (2010), performance indicators represent criteria for measuring and evaluating corporate performance, which is critical to the organization's current and future success. Performance indicators should be monitored regularly and revised if they do not meet the goals (Arif & Smiley, 2004). For educational institutions, determining performance indicators should include all stakeholders of the organization and have a direct impact on the basic budget (Conlon, 2004). According to Burke and Minassians (2002) the use of performance indicators is important because it is a real test of accountability for how well universities meet the needs of students, governments and society.
According to Arif and Smiley (2004), the key points of key performance indicators should be as follows: The number of patents, number of companies consulted, number of students working in companies, income generated for the university, number of faculty members participating, number of publications published, facultyindustry partnerships, etc.
Different universities can use different criteria in terms of performance indicators. Ohio State University has also developed key performance indicators in areas such as academic excellence, outreach programs, participation in social activities, financial resource management, student diversity, and student learning (Ohio State University, 2019). Rhodes University monitors its performance indicators under four main headings (Rhodes University, 2019): 1. Business income rate: Resources per student, faculty resources, and financial issues including debt burden rate and service expenses, etc.

General education experience:
Evaluation of basic education experience by students, employment rates of graduates, etc. 3. Graduate student placement rates: Undergraduate graduation rates, student enrollment rates for graduate programs, etc. 4. Human and organizational development perspective: The number of activities carried out for the development of students, the rates of students actively participating in these activities. Burke and Minassians (2002) suggested using 14 general core indicators to reflect the priority priorities of state policymakers. According to the study, the most critical suggestion is to enable institutions to report on internal performance. While most of these reports are not included in a government report, internal institutional reports will bring "accountability of performance reporting to units that have the most roles in producing results in most of the indicators." This will increase both internal and external accountability of performance criteria.
In a study conducted by Terkla (2011) Pfeffer and Sutton (2006) emphasized the importance of reliable data to be collected in the process of determining performance indicators. In order to increase the validity and reliability of the universities, universities improve the evidence-based management and data of the internal processes; it should combine with external data such as funds, donations and institutional collaborations. In this way, the process can be followed more detailed with periodic and annual development reports. In the strategic planning guide published by the Ministry of Development (2018b) for universities, it was stated that universities should create their own databases for performance indicators without data. It is also necessary to determine how data will be obtained for performance indicators in strategic plans. The data source may be existing operational systems or new sources to be provided through surveys, focus group studies, interviews and observations.

Method
The purpose of this study is to develop performance indicators for the strategic goals and objectives in the strategic plans of public universities by examining the domestic and foreign literature.  To determine the investment, policies, and priorities for the physical and technological infrastructure in accordance with the target growth. F4. To develop the budget use and control systems to ensure the financial and administrative discipline.

Stakeholder
SH1. To enhance the stakeholder satisfaction (student-academic-administrative graduate-personnel-board of trustees) SH2. To provide effective consultancy and guidance services for the students SH3. To create a network of healthy and continuous relationships with our stakeholders. SH4. To reward the successes achieved by the most qualified students in our country with financial support to bring them into our university and make this sustainable. SH5. To raise such graduates that will be qualified as globally preferable. SH6. To create such environments that will support the social and academic developments of the students. SH7. To contribute to the solutions of regional and national problems by offering the education and service potential of the university in favor of the stakeholders through the cooperation between the university and stakeholder (industry, supplier, society, etc.).

Learning and Development
LD1. To create and maintain a qualitatively and quantitatively competent academic staff. LD2. To strengthen and maximize any and all infrastructure that will encourage and support the scientific production, and make this sustainable. LD3. To enhance the job satisfaction of the academic and administrative personnel and to support their academic and social development. LD4. To create and maintain the innovation and intrapreneurship culture. LD5. To establish national and international relationships, and to encourage the national and international mobility of the internal stakeholders (students, academicians, and other personnel) LD6. To create the quality culture (in policy and practice: quality assurance mechanisms, processes, data collection, assessment, improvement) LD7. To ensure improvement and sustainability in academic and administrative management.

Internal Processes
IP1. To make the performance assessment efficient, and to support through reward/incentive system. IP2. To create a quality assurance system in academic and administrative processes, to ensure them to be internationally accredited, and make these sustainable. IP3. To enhance the efficiency and effectiveness of the services offered by improving the integrated management information system and data processing infrastructure. IP4. To create a corporate culture that reflects the core values adopted by the university to the fullest extent as well as an efficient and effective management structure. IP5. To create the balance between workload and manpower. IP6. To create qualitatively and quantitatively sufficient physical and social environments that will enhance the motivation of the university personnel, and make these sustainable ER1. To encourage publishing and research operations by installing a performancebased academic assessment system ER2. To ensure that technology-aided innovative learning methods to optimize the learning and to support the learning process with technology. ER3. To create and continuously update competitive, flexible programs (graduate,

Education and Research
undergraduate, associate degree, certificate) in accordance with the social expectations and industrial, national and international trends. ER4. To integrate the curriculums with the national and international programs, and to make the academic programs gradually accredited. ER5. To increase the number of original and innovative scientific studies and publications. ER6. To encourage and increase the formal and informal educational activities for entrepreneurship and innovation. ER7. To increase the academic collaborations through national and international universities, research centers, and research networks. Institutional Image IM1. To improve the "innovative and entrepreneur" university image. IM2. To ensure the national and international recognition and preferability of the university. IM3. To install the Corporate Communication/Promotion System, and institutionalize the promotional operations. IM4. To establish and maintain strong relationships with the international higher education organizations and associations (being a member to the European University Association (EUA), etc.). IM5. To qualitatively and quantitatively increase the number of events such as symposium, congress, panel, etc. to be held internationally, and ensure the attendance to them.
In this study, it is aimed to create a sample performance indicator list for universities provided that Özdemir and Tüysüz (2017) have developed strategic goals for the universities. Within the scope of the research, the strategic plans of many universities operating at Turkey and abroad were examined. 11 different universities were selected through purposive sampling in the criteria of expressing their performance indicators in their strategic plans. In addition, Strategic Planning Guide for Public Administrations (Ministry of Development, 2018a), Strategic Planning Guide for Universities (Ministry of Development, 2018b), YÖK Private Foundation Universities (YÖK, 2019) and YÖK Academic Evaluation and Quality Improvement Commission Report (YÖDEK, 2007) publications and researches were examined.
The list of universities whose strategic plans are examined is as follows: In the first stage, the strategic plans of 11 universities, the reports of YÖK and the Ministry of Development were examined and a performance indicator pool consisting of 300 criteria was created. In the second stage, in order to narrow the pool of performance indicators; expert opinion received with 2 faculty members working in the field of educational management, managerial positions in higher education institutions, previously worked in the field of strategic planning and researches on related topics. As a result of the interviews, 300 performance indicators were reduced to 45 performance indicators and grouped under 6 main strategic dimensions. The performance indicators and strategic dimensions created are given in Table 3. Number of faculty members going abroad for research or education / Total number of faculty members I3.
The number of guest researchers from abroad and the satisfaction rate of these researchers I4.
The number of promotional activities carried out at national and international level for university candidates I5.
Number of foreign graduate and undergraduate students / Total number of students I6.

Number of activities for graduates
In the third and final stage of the research, performance indicators were classified according to their importance by using AHP (Analytical Hierarchy Process) method, which is a multi-criteria decision making technique. The AHP study held administrative duties in the field of strategic planning and was conducted with 10 academics who have scientific studies in this field. Saaty (1980) for the solution of complex problems. AHP is a measurement theory for binary comparisons based on expert opinions. Comparisons are made using a scale that includes absolute judgments that show how dominant a particular element is to another, according to a particular feature. Unlike other multicriteria decision-making methods, AHP compares the criteria in pairs and measures whether the comparisons are consistent (Tayyar et al., 2014).

AHP (Analytical Hierarchy Process) is one of the Multiple Criteria Decision Making (MCDM) methods developed by
The process of creating a hierarchy with AHP is expressed as in Figure 1 (Karakaya, 2019):

Figure 1: Hierarchy Creation Process
The steps to be followed through the implementation process of AHP are as follows (Saaty, 1980): Step 1: A comparison matrix is created where binary comparisons will be made. Each participant completes its assessment according to the AHP Significance Scale developed by Saaty (1980). The Importance Scale and the values that each item can take are as in Table 4 (Saaty, 1980): When factor 1 is more important than factor 2 5 When factor 1 is very important than factor 2 7 The fact that factor 1 has a very strong importance compared to factor 2 9 The fact that factor 1 has absolute superior importance compared to factor 2 2, 4, 6, 8 Intermediate values According to the AHP Significance Scale, the participants evaluate the factors mutually and obtain binary comparison matrices so that the importance of the priorities in each component is determined. As a result of the comparison, a square matrix is obtained in which the values on the diagonal are equal to 1 (Önder and Önder, 2018): Step 2: The created comparison matrix is normalized (standardized). For the standardization, column totals are taken and each value is divided by its own column total. Thus, a standardized matrix is obtained.
Step 3: By taking the arithmetic average of the line elements in the normalized matrix, the criteria weights are obtained.
Step 4: At this stage, the consistency rate (CR) is checked. Since the AHP method reflects the evaluations of decision makers, some inconsistent rates may arise. The consistency rate should be calculated to determine these inconsistent rates in AHP. At this stage, the Consistency Ratio (CI), that is, the index value should be calculated. The formula used for the consistency ratio value is as follows. = λ − -1 In the formula above, CI indicates the consistency index, λ is the largest eigenvector in the matrix, and indicates the number of elements in the matrix. Another value required to calculate the consistency rate is the Randomness Index ( ). To get the value of RI, the value corresponding to n is taken in the random index table (Önder & Önder, 2018). Random Index values are given in Table 6 (Saaty, 1980): Finally, the "Consistency Rate (CR)" is obtained by the ratio of CI to RI. In AHP applications, the fact that CR is less than 0.1 indicates that the application is consistent (Saaty, 1980). After the analysis was completed, the responses of the participants whose CR value was above 0.1 were removed from the analysis and the analysis was continued with those whose consistency value was appropriate.

Results
The AHP results obtained for each sub-dimension of the performance indicators are as follows:

Findings Related to Finance Sub-Dimension
With the answers of 6 participants whose CR consistency values were below 0.1 in the finance sub-dimension, AHP analysis was completed and shown in  The revised items obtained by calculating the geometric mean of the criterion weights of 9 items in the finance sub-dimension is as follows:

Revised Performance Indicators for Finance Sub-dimension
1 The ratio of TÜBİTAK project revenues to total income 2 Ratio of international project revenues to total income 3 The ratio of research and development income to total income 4 The ratio of project revenues carried out in the technopark to total income 5 Number of incomes from student enrollments to total income (private foundation universities) 6 Ratio of other public and private sector financed project revenues to total revenue 7 Occupancy rates by years (private foundation universities) 8 The ratio of donation income to total income 9 Ratio of rental income to total income

Findings Related to Stakeholder Sub-Dimension
With the answers of 5 participants whose CR consistency values were below 0.1 in the stakeholders subdimension, AHP analysis was completed and shown in  The revised items obtained by calculating the geometric mean of the criterion weights of 10 items in the finance sub-dimension is as follows: The number of technoparks, socioparks, application research centers and the rates of students served in these institutions 7

Revised Performance Indicators for Stakeholder
The ratio of the number of disabled-friendly buildings to the total number of buildings 8 Number of cooperation protocols with institutions (Public and private institutions) 9 Number of club activities carried out by student communities 10 Participation rate of students in social responsibility activities

Findings Related to Internal Processes Sub-Dimension
With the answers of 7 participants whose CR consistency values were below 0.1 in the internal processes subdimension, AHP analysis was completed and shown in Table 10 (CR/ K1   The revised items obtained by calculating the geometric mean of the criterion weights of 6 items in the finance sub-dimension is as follows: Revised Performance Indicators for Internal Processes Sub-dimension 1 Accredited department / program ratio 2 Number of students / academic staff ratio 3 Average hours per week per instructor 4 Number of in-service training activities and the proportion of staff participating in these activities 5 Proportion of undergraduate students graduating in normal period 6 Number of social, sportive, cultural activities organized for the personnel

Findings Related to Education and Research Sub-Dimension
With the answers of 5 participants whose CR consistency values were below 0.1 in the education and research sub-dimension, AHP analysis was completed and shown in Table 11 (CR/ K1: 0,02; K2: 0,09; K3: 0,08; K4: 0,09; K5: 0,08). The revised items obtained by calculating the geometric mean of the criterion weights of 9 items in the finance sub-dimension is as follows:

Findings Related to Institutional Image Sub-Dimension
With the answers of 7 participants whose CR consistency values were below 0.1 in the institutional image subdimension, AHP analysis was completed and shown in  The revised items obtained by calculating the geometric mean of the criterion weights of 6 items in the finance sub-dimension is as follows: The number of guest researchers from abroad and the satisfaction rate of these researchers 6 Number of activities for graduates

Conclusion
According to the finance sub-dimension AHP results, first 3 dimensions are; TÜBİTAK received project income, international project income and research and development income. The first dimension for private foundation universities is the income from student enrollments. Donation and rental income remained at the top.
In public universities, all of the staff salaries and substantial expenses are covered by public resources. However, in line with the Public Financial Control and Management Law No. 5018, it is stated that performance-based budgeting will be implemented in public institutions, including universities (YÖK, 2015). In this context, universities need TÜBİTAK projects and projects supported by international funds etc. in order to increase performance quality and access different financial resources.
"Stakeholder satisfaction and satisfaction survey results" rank first in the stakeholders sub-dimension. Today, with the spread of accreditation processes and quality assurance systems, many universities are developing various strategies to measure and evaluate the satisfaction of stakeholders. For example, Aydin University (2020) stated all the details of the process in the "Stakeholder Satisfaction Policy" on its website, Ankara University (2020) announced the "Strategy Development Department External Stakeholder Satisfaction Survey" online. In the stakeholder sub-dimension, the second item with the highest criterion weight is the employment rate of graduates. Employment rates are one of the most important criteria for parents and students in university preferences (Soutar & Turner, 2002;Moogan & Baron, 2003;Uncle, 2011). Universities provide opportunities for students to post-graduate job opportunities, internships, 2nd foreign language, etc. and they should be supported by various courses and certification.
"The ratio of the number of academic staff sent to universities abroad to the number of academic staff in the first place in the learning and development sub-dimension"; the second rank is "The rate of students participating in the exchange program." Internationalization in higher education is one of the topics that YÖK has emphasized most recently. YÖK published the Internationalization Strategy Document in Higher Education -2022(YÖK, 2017 in order to guide universities in this field. One of the strategy mentioned at the document is "Turkey is providing to become a center of attraction in the higher education area" which has "Objective 1.4: Increasing the number of participants in international student and faculty exchange in the field of cooperation and exchange programs". In this context, the rates of faculty and students participating in exchange programs are an important performance indicator for universities. In the sub-dimension of internal processes, the items with the highest criterion weight are "Accredited department / program ratio," "Number of students / faculty ratio" and "Average weekly hours per academic staff member." The findings obtained are in line with the Strategic Planning Guide for Universities (Ministry of Development, 2018b) and the accreditation, academic staff / student ratio and faculty course load, which the recent YÖK emphasizes. Prominent indicators in the education and research sub-dimension are: "Total number of publications and scientific activities per faculty member," "The place of the university in the national and international academic ranking" and "Number of full-text publications per academic year." In this context, universities require more faculty members to implement various incentive programs and arrangements that will increase the broadcast performance and increase the institution's position in the national / international rankings. Similarly, from the related findings, it is concluded that university administrators give importance to "Number of teaching staff who go abroad for research or education" and "Number of promotional activities carried out at national and international level for university candidates" in order to strengthen the institutional image.
According to the AHP analysis conducted at the end of the study, a set of 45-item performance indicators, which are ranked according to criterion weights under 6 main subtitles, namely financial, stakeholders, learning and development, internal processes, education and research and institutional image. It is seen that the findings are in line with the recent the Council of Higher Education policies, Strategic Planning Guide for Universities by the Ministry of Development (2018b) and the Higher Education Council's 2016-2020 Strategic Plan (YÖK, 2015).

Recommendation
Conducting different researches for each performance indicators sub-dimension i.e. stakeholders, education research, learning and development will make important contributions to the field and strategic planning topics. Income from student records in the financial resources of foundation universities is an important income item. In addition, private foundation universities donate income, dormitory income, etc. It differs relatively from state universities in matters. At this point, the performance indicators in the financial sub-dimensions should be supported by independent or comparative research, based on the university relations with the owner foundation.
When the strategic plans of the universities are examined within the scope of the research, the number of publications in general, the ratio of accredited departments, international projects etc. performance indicators appear to be in the first place. However, especially when examining the strategic plans of well-known universities around the world, besides physical or educational strategic goals, it is seen that more inclusive strategic goals are included. For example, "Raising the minds that will inspire the world at Yale University (Yale University, 2018)" or "Making Oxford experience the best experience in the student's life (Oxford University, 2018)". These universities also include performance indicators including a series of activities and practices in order to achieve the objectives mentioned in their strategic plans. Universities operating in Turkey should complete basic indicators rapidly and spend time to actualize more visionary strategic goals.
Strategic planning became widespread in Turkey in recent years and a new legislation is mandated for by public administrations. At this point, significant deficiencies are noticed when the strategic plans and performance indicators of the universities are analyzed. Even creating performance indicators based on concrete data such as measurable and ratio / number, which is one of the most basic principles, has been neglected in some strategic plans. In this context, YÖK should increase the number of guide activities and documentation to universities, especially in printed publications, video recordings and university trainings, on strategic planning and performance indicators. Under the leadership of YÖK, policy makers should initiate such a planning and create a weighted list of ranking criteria under various subtitles. In this way, universities will be able to make adjustments according to these criteria while creating performance indicators.