Poor Results in the Evaluations of the Programme for International Student Assessment (PISA) and the Unsuccessfulness in the Evaluation of Public Policies for Education

The news on Brazil’s low ranking in each edition of the Programme for International Student Assessment (PISA) tests creates intense debate about the quality of education in the country, portrayed by the mainstream media, whose discourse will be analyzed herein in order to exemplify contradictions that may become obstacles to the improvement of these indicators. What prevails is an interpretation of the results of standardized educational assessments that holds students (and by association their teachers and their schools) accountable for the poor results in these tests. Although progress has been made in terms of democratizing the entrance of the Brazilian population in schools, said approach compromises students’ possibility to remain in school, as well as their access to a quality education through an educational policy perspective that prioritizes social reception to the detriment of the commitment with knowledge transmission. In the discourses analyzed in this work, we notice there is no emphasis on the fact that international assessments, such as the PISA, should be used to measure the efficiency of a country’s public policies for education instead of individuals. When one simplifies the analysis of these assessments’ results through a discourse that stresses only the measurement of students’ performance, one obliterates the Government’s duty to ensure quality education. Said movement is justified by a psychologizing interpretation of human development as a mechanism for segregation and pedagogical exclusion.


Introduction
The disclosure of the data analyzed in each edition of the Programme for International Student Assessment (PISA) in Brazil is widely covered national press. When we search for news headlines in newspapers or other digital platforms on the subject, we find emphasis on the poor results obtained by Brazilian students, such as: "Brazilian students perform poorly, According to Pisa" (Record TV 2018), "Brazilian students will take 260 years to reach rich countries' reading index, says World Bank" (G1 2018), "Emotional unpreparedness can jeopardize Brazilian students as much as lack of knowledge" (Idoeta and Guimarães 2018), "Brazilian students like science, but are overwhelmed by its content" (Righetti 2016).
The reading of these texts leads to individualizing a complex issue; the body of such articles has many references to a supposed cognitive demotion of Brazilian students, which would result in their poor performance in the programme, and ultimately be related to the limitations arising from low socioeconomic conditions. However, paradoxically, little or no question is asked about the public policies implicated in the poor performance of our students, which directly impact the quality of the education offered. How can students be assessed on content and concepts not taught to them? We will seek to highlight in this paper how standardized assessments such as PISA should produce indicators for policy making and school management; and an ambiguous speech in the documents produced by the Organization for Economic Co-operation and Development (OECD) gives room for the overriding accountability of students (their families, their teachers) for school failure, silencing and exempting the State from the duty, secured by law, to provide conditions for access to quality education for all citizens.

Pisa and the evaluation of education systems (and not student evaluation!)
In the document PISA for Development Assessment and Analytical Framework: Reading, Mathematics and Science (OECD 2018a) we find that PISA is an indicator of a commitment made between OECD member and partner countries to monitor the result of education systems (and not the cognitive capacity of students from different countries ...) and facilitate the sharing of political actions. According to the text, PISA's focus should be on building the capacity of participating countries to implement large-scale international education assessments, using their results to support evidence-based policies. We highlight how much Pisa's results should be interpreted to support the assessment of a country's education policies. On the website of the Brazilian Ministry of Education (MEC 2019) PISA's goal is described as follows: Pisa's goal is to produce indicators that contribute to the discussion of the participating countries' education quality to support policies which improve primary and secondary education. The assessment seeks to ascertain to what extent the schools of each participating country are preparing their youth to act as citizens in contemporary society.
However, on the same platform, we find PISA defined as an "International program that assesses the capacity of Brazilian students" (MEC 2019, emphasis added). In fact, in the various tabs of our Ministry of Education's digital platform dedicated to PISA, there are no documents that analyze Brazilian educational policies and correlate them with the results obtained in the evaluation. There are, however, reports on student performance and a discourse on the importance of developing several skills and competencies, which would improve exam results. We think there is an implicit premise here that understands such skills and competencies from an innate developmental perspective, in a bio-psychomaturational logic, unlike the historical-cultural approach advocated by theorists such as Vygostky (1999), who establishes a correlation between intrapsychic development derived from interpsychic relationships, in which development would originate from learning processes.
PISA's creators understand that the exam should be a demonstration of what citizens in different countries know, and how they apply and extrapolate the knowledge passed on to them, going beyond its mere reproduction. However, how can one apply and extrapolate knowledge in a context such as ours in Brazil, where, for example, many high school students spend months in school without a math teacher? Or are systematically dismissed from classes because of a lack of teachers? Or who can't attend school because it lacks water or toilet paper? How can we measure extrapolated and applied knowledge in a context where learning is absent or fragile? Our education system is divided and uneven, with a few students from socially and economically advantaged families being able to attend high-tuition schools focused on knowledge transmission and use of technological resources, while the majority of the population attends public schools associated with poor educational quality.

Brazil in PISA
In the 2015 edition, of the seventy countries that participated in PISA, Brazil ranked 63 rd in science, 59 th in reading and 65 th in mathematics. In the OECD documents on the indicators of our country in Pisa (OECD 2017) we find that students' performance (and not Brazilian educational policies!) is below the average of students from countries associated with the OECD in the three areas that make up the evaluation. In spite of the increase in annual spending per student in Brazil (US$ 3.8 thousand, although the average spending of OECD member countries is almost threefold; US$ 9.0 thousand), this increase did not significantly impact students' exam results and increase in learning, which leads us to conclude that such investments have not been applied to effectively improve teacher training policies and enhance teaching practices.
The alleged democratization of school access in Brazil brought the great advantage of virtually universalizing enrollment of school-age children and adolescents, especially from a portion of the population in an income class that for years has been left out of school. However, we still face many challenges; the main one is that access to school is not the same as access to knowledge through education.
We still face high grade repetition rates (36% of 15 year olds have repeated at least one grade), and high levels of reproduction of social inequalities in school, in an education network divided into public and private, with many discrepancies in the quality of education offered in different institutions. There is also a large gap between the age of students in relation to the grade attended, many difficulties of concrete conditions and educational resources to perform pedagogical activities.
All these aspects are indicated in the reports presented by the OEC/MEC on Brazil's performance in PISA, but are not evident when the results are analyzed, ultimately leading to focus on the approach that bad indicators would be a consequence of the intellectual inability of our students (as if the conditions mentioned above did not have a significant impact on student learning).
The silencing and exemption from a public discussion on the evaluation of public education policies in Brazil comprise a perverse mechanism that blames individuals for the failure of State educational programs, as expressed, for example, in the statement of students, in a newspaper article we analyzed: After learning about the existence of Pisa, the selected students said they were anxious. "It makes me proud, but it also is very big responsibility," said Isabela Jesus, 15. "If the grade drops this year, I'll be kind of upset because I'll think I'll have contributed to it," adds schoolmate Letícia Prado, also 15 (R7 Educação 2019).
Isabelas and Leticias and so many young people who take the PISA every three years (as well as many teachers) are unaware of the exam and its purpose, but are affected by the insistence on the association between Brazil's performance in the exam and the supposed low cognitive ability of our students. However, as we will discuss further on, the construction of cognitive tools allied with affective energy depends strongly on educational processes. The skills and competencies tested in (standardized or non standardized) assessments are not innate or spontaneous, but rather built on the dynamic between learning and development. But we insist: there is no epistemological construction without education! The OECD document in conjunction with MEC (OECD, MEC 2016) provides an overview of the "strengths and weaknesses of Brazilian students in PISA 2015" (p. 51), and a certain tendency to consider student's "prior knowledge" as a crucial element for conducting the exams, but in a spontaneous manner, and that obliterates the importance of education. "The reader produces meaning as a reaction to the text, using prior knowledge and textual and situational clues that normally derive from his or her society and culture." (OECD, MEC 2016, 93). In addition, we cannot fail to indicate that the very small use of public spaces and cultural apparatuses by children and adolescents from disadvantaged socioeconomic conditions makes it difficult for our students to learn even in nonschool contexts.
We understand that PISA's creators assume a (implicit?) premise that education took place, so students (and the educational policies of their countries) could be evaluated under similar conditions in different countries. However, in the case of Brazil, the very document produced by the OECD/MEC affirms the fragility of the educational processes, which would put our students in a different condition when compared to students from other countries, especially those that showed the best indicators in the evaluation: The level of difficulty shown by Brazilian adolescents in items whose texts represent educational situations suggests that our schools, generally speaking, do not develop systematic work with the didactic genres. On top of that, schools have not yet assumed that teaching how to read -as well as how to write -must be a commitment of all areas. Therefore, it is not enough for Portuguese language teachers to invest in reading taking into account personal, public and occupational variables. Students need to also be able to read texts from an educational perspective. By building this skill, they can read to learn (Sticht, 1975;Stiggins, 1982apud OECD, 2016 at school and throughout life (OECD, MEC 2016, 108, emphasis added).
If almost all school-age Brazilians attend school, but most do not know basic mathematical operations, have little notion of scientific concepts and do not interpret written texts, we should ask ourselves; what is the function of schools in Brazil? Libâneo (1992) states that in our country our schools predominantly provide knowledge for the rich and fulfill the function of social assistance for the poor, at the expense of the commitment toward the transmission of knowledge.

Results and discussions
One of the main missions advocated by PISA is that evaluation should "help governments shape their education policy" (OECD 2018c) in order to promote equity in education.
Equity is understood here as the effort of education systems and their schools to promote equal learning opportunities for all students, regardless of their backgrounds. We should highlight here that seeking to control the learning variable once again individualizes and psychologizes a dimension that can hardly be inferred. We think that the issue of equity in education should, in turn, be guided by the promotion of equitable education opportunities and access to knowledge for everyone. We cannot guarantee that a student will learn, but we can affirm that without education the learning process is possibly compromised.
Equity as a core value should guide education policies with respect to individual, social, geographical, political, cultural and economic differences, but strictly with respect to the commitment that regardless of differences any student can have access to knowledge. According to the guidelines proposed for PISA's next versions, such access to knowledge occurs not only at school, but also through cultural, social, religious, political and economic apparatuses that impact schooling. In this sense, there is the recognition that educational policies should invest in expanding socially (and culturally) inclusive environments, the quality of education, the time students have for learning and the material resources that mediate the access to knowledge.
On the other hand, following the logic of the dual discourse that we have criticized in this paper, such crucial aspects pointed out in the documents prepared by the OECD are weakened and diluted in the emphasis given to meritocracy, translated above all into the concept of resilience. Such discourse gives rise to the idea that students, despite their poor social, economic, cultural, and educational conditions, should "strive" hard to overcome such difficulties. The document analyzed (OECD 2016) even uses the expression "social resilience", eliding the social dimension through a psychologized perspective.
PISA's results illustrate global social, economic and political inequalities. They show the fragility of democratic systems, as in the case of Brazil, where everybody is equal in the eyes of the Law, but some are more equal than others (Orwell 2018(Orwell [1945), especially those who can pay (a high amount) for quality education. Evidently the fact that our students now have access to school does not imply that they have access to knowledge. There is no fairness of opportunity in Brazil, as reflected in our low social mobility, which is by no means just a matter of merit.
Through questionnaires answered by the students who took the exam, there was also the aim to identify how psychological well-being affects their performance and their (professional) future expectations. Not surprisingly, it was noted that students designated "to be in social and economic disadvantage" express lower psychological well-being, and since "students' career expectations reflect their psychological well-being" (OECD 2016, 187), the conclusion is that these disadvantaged students (as is the case of Brazilian students) dream little about a socially valued career, and this condition produces psychological distress and a certain demotivation that would impact assessments' results. But how to build dreams with so few tools and contents? (Many students from Brazilian public schools are unaware of the existence and means of entry into competitive and renowned public higher education institutions in Brazil (in our country, many students from public universities are graduates of private primary and secondary schools). For further details, refer to the example of a project to include students from public schools in the University of São Paulo (https://pt-br.facebook.com/CUCoUSP/). We understand to be a trigger of anxiety and psychological distress a situation in which a student is required to perform well in an assessment whose content was not taught to them, an action that in turn compromises the construction of the very psychological structures required in the exams. Therefore, we understand once again that despite the recognition of how social, cultural, political and economic conditions interfere with students' performance in the exams, it reinforces the idea that such difficulties are intrapsychic when affirming for example that "student resilience -the student's capacity to overcome barriers and adversities that typically hinder learning processes and outcomesis a key issue for both equity in education and social mobility" (OECD 2018d, 98).
But how to overcome such barriers? Only with "willpower?" In this sense we affirm that it is impossible to think of resilience as a subjective condition without offering concrete social, cultural and pedagogical elements through which such psychic action can be supported.
The lack of a thorough and widely publicized evaluation of public education policies contributes to maintain psychologizing mechanism for understanding schooling processes, individualizing and reducing the complexity of the analysis of such issue. Paradoxically, the lack of commitment to the quality of education compromises the consolidation of what Vygotsky (1999) designates superior psychological structures.
This author states that it is through education, through the transmission of scientific concepts that the most complex psychological structures are constituted. No concept can be taught by simple training; learning is the result of a series of psychic operations of appropriation through language, the result of the establishment of relationships between signs and meanings. Vygotsky affirms that school learning anticipates and enhances psychological development through "vigorous mental activity on the part of the child" (Vygotsky 1999, 107). Scientific knowledge taught at school aims to develop mental operations, reflective consciousness and the deliberate control of thought. Unlike spontaneous knowledge, thanks to the understanding of scientific concepts there is the expansion of a child's power of thought, through greater attention that becomes increasingly deliberate and voluntary, and the use of a memory that becomes less mechanical and increasingly driven by meaning, and is used intentionally by children.
Vygotsky says that "the consciousness of being aware" (Vygotsky 1999, 114) is synonym of thought, and an action potentiated during schooling, where "school learning induces generalized perception, thus playing a decisive role in raising a child's awareness regarding their own mental processes (115). Therefore, it is through educational processes, along with the learning of content, concepts and theories of different disciplines that a student develops "mental faculties in general, as well as specific skills" (116).
The psychological functions required in standardized assessments are not spontaneous but rather dependent on educational processes. There is a Brazilian song by Djavan called "Esquinas" (Street Corners) with a verse that illustrates well the origin of the uneasiness caused by the disclosure of such exams' results, and which translation is: "Do you know what is not to have something, but needing to have it to give?". The evaluation indicators of Brazilian public policies will not increase significantly as long as such policies are not committed to the right to education based on the principles of equal conditions for access to knowledge and attendance in school, as guaranteed in our Federal Constitution (Brazil 1988).

Conclusions
There is a consensus between the documents produced by the OECD and the analysis produced in this paper showing that countries with the best results in PISA deliberately and consistently choose to prioritize teacher appreciation policies.
Such policies relate to teacher training, career planning, salary bonus, as well as continuing education, programs to encourage young people to become teachers, and projects to attract good teachers to more "challenging" contexts and classes.
We understand that standardized assessments such as PISA should be used to provide opportunities for students to learn, through public policy adjustments rather than as a psychometric tool that assesses individuals, which consequently does not contribute to fighting against unequal opportunities for access to knowledge.
The poor results obtained by Brazil in PISA indicate the low appreciation given in our country to educational processes (which, unfortunately, is considered as a luxury commodity, preferably offered in private institutions). The recurring complaint regarding the poor quality of Brazilian public schools and the poor performance of the students who attend it hinders the perception by students themselves, teachers and families that access to quality education (and not only to school) is a right acquired by Law in Brazil. This right is considered subjective, and any person or institution that feels harmed in relation to it can and should bring legal action against the State, claiming such right (Bautheney 2014). Unfortunately, this perspective of quality education as a subjective public right is unknown to a large part of the population, who conform to low quality education, in a context of submission that is so peculiar to the Brazilian people.