*Keywords:* mathematics literacy, thematic analysis, assessment, knowledge, teachers’ conceptions

‘milana-dabic-boricic’, ‘ivana-vulic’, ‘marina-videnovic’

---

Volume 14 (Supplement Issue), 101-121 Faculty of Education©, UM, 2020

Mathematical literacy and assessment: Differences

between the PISA study paradigm and mathematics

teachers’ conceptions

Milana Dabić Boričić^1 , Ivana Vulić^2 and Marina Videnović^3

(^1) Teacher Education Faculty, University of Belgrade; 2 Foundation Tempus, Belgrade;^3 Institute of Psychology, Faculty of Philosophy, University of Belgrade Corresponding author: milana.dabic@gmail.com Abstract : OECD/PISA 2012 survey data showed that mathematical literacy achievement in Serbia is 50 points below the OECD average, while approximately 40% of the students fall into the functionally illiterate group. At the same time, more than half of the students in Serbia reported a high level of mathematics related anxiety. A possible explanation for the low score on the PISA survey lies in the incompatibility of mathematics’ teaching practices in the Serbian education system and the functional knowledge assessed by PISA. This study aims to examine mathematics teachers’ conceptions of mathematical literacy and the assessment of mathematical knowledge. For this purpose, in-depth interviews with 15 mathematics teachers (from primary and secondary schools) and university mathematics professors (from Mathematics Faculty and Teacher Education Faculty) were performed. The participants were asked to define mathematical literacy and to express their opinion regarding the entrance exam (obligatory exam for 14-year-old students finishing primary school), the PISA survey, their assessment practice, and different types of mathematical tasks (PISA tasks and traditional tasks). The participants’ answers to these topics were analyzed using the Inductive thematic analysis to better understand the teachers’ conceptions of mathematical literacy in comparison with the PISA paradigm (the abstract level) and their perception of the differences between PISA and traditional tasks (concrete level). The analysis revealed four different teachers’ conceptions of mathematical literacy (sorted by descending frequency): 1) knowledge about basic mathematical concepts (main formulas, operations, geometry objects, etc.); 2) correct use of mathematical symbolism; 3) use of mathematics in everyday situations (such as in shops or markets); 4) developing a specific way of thinking. Teachers’ comments about the differences between the different types of tasks

highlighted additional differences. PISA tasks were described as both more interesting and complicated, as well as assessing reading ability, while the tasks from the final exam were observed as more formal, solely mathematical tasks. The teachers’ opinion was that PISA-like tasks should be introduced to mathematics practice as an addition to the traditional, solely mathematical tasks. The results reveal broad differences between the underlying logic behind PISA testing and Serbian mathematics teachers’ understanding of mathematical literacy and knowledge assessment. In order to overcome the gap between these conceptions, it is necessary to deconstruct meanings present in everyday teaching practices.

Keywords : mathematics literacy, thematic analysis, assessment, knowledge, teachers’ conceptions.

Introduction

The Program for International Students Assessment (PISA) is considered as an important tool for the assessment of a country’s quality of education. Therefore, the results of the PISA study are considered very important for participating countries, and they usually provoke large public discussions about the education system, the quality of teaching, teachers’ education, and the status in society, etc. Serbia has been a PISA participant since 2003, and all of the previous results show significant underachievement of Serbian students in all tests compared with the OECD average. The average achievement of Serbian students on the mathematical literacy test was 449 points in 2012 (Pavlović Babić & Baucal, 2013). Although that achievement was significantly higher in 2012 compared to 2009 (7 points higher) and 2006 (14 points higher), Serbian students are still a year and a half worth of education behind the students from OECD countries that score 500 points or higher (Pavlović Babić & Baucal, 2013). Furthermore, 39.9% of the students are below the second level (similarly to 2009), and only 5.7% of students are on the fifth and sixth level of achievement.

Additionally, a secondary analysis of the PISA study shows that more than half of the students experience a high level of mathematics related anxiety, i.e. they have negative reactions when dealing with mathematical concepts during evaluation (Radišić, Videnović & Baucal, 2015). Interschool differences explain only 7% of the variance – individual and school-related variables such as the interest for mathematics, school, and classroom atmosphere and ESCS can be connected to mathematics anxiety (Radišić, et al., 2015).

A possible explanation for the low achievement on PISA, especially in the mathematical domain, could lie in the noticeable differences between the approaches to mathematical literacy present in the Serbian educational system compared to the PISA conception of mathematical literacy (Pavlović Babić & Baucal, 2013). First of all, mathematical literacy is not a frequently used concept in Serbian education; instead, the concept of mathematical knowledge is preferred. Mathematical tasks used in teaching mathematics in Serbia and assessing the students’ knowledge are, to a great extent, knowledge-oriented (Radišić, Baucal, & Videnović, 2014).

Since the differences in paradigms present in the Serbian education and PISA may be one of the contributing factors of the low achievement of Serbian students on PISA, this study aims to explore in more detail the differences between these paradigms as they are perceived by mathematics teachers and university mathematics professors. Teachers’ beliefs and conceptions about the nature of mathematics are considered relevant for this study since their conceptions have been known to affect their teaching practice (Thompson, 1992). The findings indicate that the most important factors essential for the students’ contextual problem-solving are teachers’ attitudes, competencies, and deficits (Verschaffel, Greer & de Corte, 2000).

Philipp (2007) gave a proposal of descriptions/definitions which differentiate between conceptions and beliefs, making the beliefs “Psychologically held understandings, premises, or propositions about the world that are thought to be true” and conceptions “a general notion or mental structure encompassing beliefs, meanings, concepts, propositions, rules, mental images, and preferences” (p. 259). According to this definition, we have chosen the term conceptions, which we find more appropriate for this study since we will be examining the teachers’ point of view regarding relevant mathematical knowledge, the process of gaining that knowledge, the manner in which it should be assessed, different existing mathematical conceptions, etc. However, before we move to the aim of the study, we will shortly present the main concepts relevant to the understanding the aforementioned paradigms.

Mathematical literacy and assessment

The key concept for this study is the already mentioned concept of literacy (in this context, we are explicitly referring to mathematical literacy), which is the basic concept in PISA but also an often mentioned concept in modern literature and educational practice.

Literacy is not a new term; on the contrary. In the field of education, literacy has been present for some time. It is closely related to the notion of competencies that represent a set of knowledge and practical skills, including the corresponding values and attitudes that enable a person to solve specific problems (see OECD, 2005; Rajović & Radulović, 2007). Literacy relates to the set of skills and knowledge that comprise basic competencies in a particular field or domain (such as reading, mathematics, and science).

PISA defines literacy as knowledge and skills that are essential for future life (OECD, 2006). Although PISA does not diminish the importance of knowledge gained from school curricula (OECD, 2006), the emphasis is on adopting a more extensive range of concepts and skills. Thebroad, generalized knowledge and skills that PISA assess, include communication, adaptability, flexibility, problem-solving, and the use of ICT skills that are developing through different curriculum contents, and their assessment requires a broad, cross-curricular approach (OECD, 2006).

Literacy primarily referred to the ability of the individual to understand, evaluate, and use written texts in order to participate in society successfully, achieve goals and develop his/her own knowledge and potential (Group, 2009; OECD, 2016). Nowadays, in addition to the reading literacy, the PISA study recognizes the importance of literacy in other domains, primarily mathematical and scientific, and, more recently, in solving problems and the financial domain (OECD, 2013). Therefore, literacy within the OECD conceptual framework is broadly defined as understanding and managing processes that are at the core of a domain, the acquisition of basic knowledge and skills necessary for process management, and their application in specific contexts (OECD, 2000).

In the OECD paradigm, mathematical literacy is referred to as the capacity to identify and understand the role that mathematics plays in the world, to make well-founded judgments, and to use and engage with mathematics in

ways that meet the needs of one’s life as a constructive, concerned, and reflective citizen (OECD, 2005; Pavlović Babić & Baucal, 2013). Mathematical literacy is not a term used only in PISA. In the paper Understanding Mathematical Literacy: The Contribution of Research , Kilpatrick (2001) explains that many terms could be used for “successful mathematics learning”: mathematical proficiency, numeracy, mastery of mathematics, mathematical competency, and mathematical literacy. According to this author, mathematical proficiency , which he uses synonymously with the term mathematical literacy , has five strands: “(a) conceptual understanding , which refers to the student’s comprehension of mathematical concepts, operations, and relations; (b) procedural fluency , or the student’s skill in carrying out mathematical procedures flexibly, accurately, efficiently, and appropriately; (c) strategic competency , the student’s ability to formulate, represent, and solve mathematical problems; (d) adaptive reasoning , the capacity for logical thought and for reflection on, explanation of, and justification of mathematical arguments; and (e) productive disposition , which includes the student’s habitual inclination to see mathematics as a sensible, useful, and worthwhile subject to be learned, coupled with a belief in the value of diligent work and in one’s own efficacy as a doer of mathematics” (p. 107). The last two aspects present in Kilpatrick’s definition emphasize logical thought, justification of mathematical arguments, and seeing mathematics as a useful subject, which is very similar to the OECD definition, but the latter has wider meaning emphasizing the “use and engage[ment] with mathematics in ways that meet the needs of one’s life as a constructive, concerned and reflective citizen” (OECD, 2005).

Jablonka (2003) approached with the concept of mathematical literacy by arguing that the concept is connected to a particular social practice and represents the relationship between mathematics, the surrounding culture, and the curriculum. The concept can change depending on the rationales and values of its proponents, which can be measuring the output of formal mathematics education, popularization of academic mathematics, vocationalizing the general mathematics education, or educating critical citizens (Jablonka, 2003); the latter being the most connected to the OECD concept of mathematical literacy. However, Jablonka provided solid arguments for her view that the OECD definition of mathematical literacy is not cross-cultural because it depends on the economic and technological development of a country and the lifestyle of its inhabitants. It is hard to imagine, she argued, that the pollution, traffic safety, or population growth

are ‘real-life’ situations in all countries. The ideological, political, and economic conditions in a country modify the influence that mathematical literacy has on “individual’s success in the workforce,” “raising the mathematical literacy standards of a population (and thereby enlarging the mathematically skilled workforce,” and “[rising] the material and economic (let alone the democratic) development of a country” (p. 81). Additionally, she argued that it is very challenging to define mathematical literacy in the first place because of the abstract nature of mathematics. Literacy is based on a learned system of symbols and reasoning with the aim to represent something outside this system, which is hard in mathematics because of its highly decontextualized nature.

Word problems and the modeling process The OECD concept of mathematical literacy is operationalized through the PISA study, which contains textual tasks illustrating the specific context of some kind, which is very relevant for solving an actual task. The task itself requires solving some realistic problem by using mathematical knowledge. This kind of task can be referred to as a word problem.

Word problems are often defined as “verbal descriptions of problem situations wherein one or more questions have raised the answer to which can be obtained by the application of mathematical operations to numerical data available in the problem statement” (Verschaffel, Depaepe & Van Dooren, 2014, p. 641). In the traditional instructional practice, mathematics teachers use stereotyped word problems that can be solved by carrying out arithmetic operations with the given numbers (Verschaffel et al., 2000). Hence, the students comprehend word problems as algorithmic tasks and solve them without taking into consideration the context of the problem (De Corte, Verschaffel & Greer, 2000).

In the last decades of research in mathematical education, great importance was given to solving problem situations in the real world, which is sometimes identified with the modeling process (De Corte et al., 2000). An investigation of the modeling process by Blum and Leiss (2007) resulted in a seven-step modeling cycle: 1) Constructing; 2) Simplifying/Structuring; 3) Mathematizing; 4) Working mathematically; 5) Interpreting; 6) Validating; and 7) Exposing. According to these steps, the construction of the situation model comes first, which means that the student has to understand the situation of the problem. Then, the situation needs to be simplified, which leads to the real model of the situation. Mathematizing means that the real

model is transformed into the mathematical model. Working with a mathematical model gives mathematical results that are interpreted in the real world, validated, and exposed.

Figure 1. The main features of the PISA mathematics framework (OECD, 2013, pp. 37)

A similar model to the one developed by Blum and Leiss is present in the PISA mathematical framework presented in Figure 1. The fundamental mathematical capabilities that underpin the framework are Communication; Representation; Devising strategies; Mathematization; Reasoning and argument; Using symbolic, formal, and technical language and operations; Using mathematical tools, while the main processes are: Formulate; Employ; Interpret/Evaluate.

Although not all the steps are the same, the key capabilities needed for solving word problems are present in both models. Understanding the steps/capabilities that underpin the solving process is crucial for the teaching practice since they help not only to teach the students relevant skills but also to understand better the types of difficulties that may occur during the learning process and to address them accordingly with specific teaching methods and techniques. Blum and Boromeo Ferri (2009) presented the research results that support the hypothesis that mathematical modeling can be taught and learned, but “teaching effects can only (to be more precise: at most) be expected based on quality mathematics teaching” (p. 52). This finding supports the general idea behind the OECD concept of literacy, which is that literacy, in its nature, can be thought and learned. The key question is, of course, how it can be thought.

Concept of mathematical literacy in the Serbian educational policy and practice

In the Serbian education, the concept of literacy is still new. Educational reforms, which started almost a decade ago, have acknowledged the notion of competencies. Accordingly, the latest initiatives aim to change the focus of education from the content that needs to be thought to the outcomes that need to be achieved. One of the most recent policy documents, Regulation on General Standards of Achievement – Educational Standards for the End of Compulsory Education, 2010 , define the educational standards of students’ achievement for each subject at three levels (basic, middle, and advanced). In the field of mathematics, these standards are defined by the mathematical area, e.g., geometry, arithmetic, and, on the third level, they include knowledge about mathematical concepts, the application of mathematical formulas (in realistic problems, when data are not explicitly provided), solving complex textual tasks, etc. The expected outcomes may vary depending on the specific field, and they are mostly defined by the content.

The mentioned standards of achievement serve as guidelines for teachers’ practice – both the teaching and assessment practice. The overall measurement of the achievement of outcomes recognized in the Standards in the area of mathematics (as well as mother tongue and science) is an obligatory final exam, which is also the entrance exam for the majority of secondary schools. This exam consists of an average of 20 mathematical tasks that assess the students’ knowledge of mathematics taught during primary education. Both tasks in the final exam and the mentioned standards represent the paradigm of mathematical knowledge present in the Serbian educational system.

Rationale for the study

Based on the view that the conception of mathematical literacy is related to practice (Jablonka, 2003), we wanted to examine the teachers’ conceptions of literacy from the direct and indirect angle. Firstly, we wanted to obtain their description of mathematical literacy. Secondly, we wanted to gain an insight into their conception of literacy from their view of assessment in mathematics, view of the final exam and PISA assessment, and the analysis of different tasks. With regards to the second, indirect angle, we began from the assumption that the teachers’ notion of mathematical literacy could have two

opposite ends. One is supported by the tasks from mathematics textbooks and the final exams. By including this kind of tasks exclusively into their practice, the teachers set the cornerstone for the view that mathematics is based on abstract, generalized concepts, and that, therefore, mathematical literacy is the level of generalization of the concepts, proficiency in the procedures and the extent of mathematical knowledge. On the other hand, we have a qualitatively different conception of tasks present in PISA, TIMSS – contextualized application of mathematical knowledge. Such tasks support the already mentioned OECD conception of mathematical literacy.

This research aims to examine the teachers’ conceptions of mathematical literacy and assessment of mathematical knowledge through the prism of PISA tasks and tasks from the final exams, which represent the mentioned concepts of mathematics and mathematical literacy. We posed the following research questions:

1. How do teachers define mathematical literacy, and are there differences in the primary and secondary school mathematics teachers’ and university mathematics professors’ definitions?

2. How do teachers perceive the assessment of students’ knowledge/ mathematical literacy – what should be assessed and in what way? Are there any differences between primary and secondary school teachers and university professors regarding this aspect?

3. What are the teachers’ opinions regarding the PISA test, the final exam, PISA, and the final exam tasks? Is there agreement with their definition of mathematical literacy?

Method

The procedure and used materials: The method used in this research was the semi-structured interview led by a trained examiner. The interview consisted of seven general topics:

 Concept of mathematical knowledge/literacy (teachers’ determination of the concepts, their view of the process, etc.)  Assessment practices (teachers’ views of the grades and competencies needed for each grade)  View on the teaching materials (books and materials they use and what for)

 View on the final exam (advantages, disadvantages, and improvement)  View on the PISA assessment (discussion of the general concept and analysis)  View on the differences between the PISA and final exam tasks (task difficulty, attractiveness, familiarity, etc.)  View on the future development of the final exam (after taking a look at the presented exams)

Figure 2. Pair of tasks from PISA and final exam (content domain functions), presented to the interviewed teachers

Additionally, the view on the differences between the PISA and final exam tasks was examined by asking the teachers to give their opinions about the three pairs of tasks. One of the tasks in the pair was from PISA, while the

other was from the final exam. The tasks in each pair were from the same content domain (Figure 2). The interviewer presented the pair of tasks to the teachers/professors, gave them time to think, and then asked them questions about the tasks. The general questions were: Which task from the pair would the students solve more successfully?; Can you estimate the percentage of the students who would be able to solve the task?; Which task would be more interesting to the students?; Could this be an adequate task material for the textbooks? The participants’ responses were recorded, transcribed, and analyzed using the inductive thematic qualitative analysis (Fereday & MuirCochrane, 2006).

Participants The participants in this study were 12 primary and secondary school teachers and three university mathematics professors. Six out of teachers teach mathematics in a primary school (to 10-14 year-old students), while four teachers teach mathematics in a secondary school (to 14-18 year-old students). Two university professors were experienced in teaching both primary and secondary school students. Two university professors are associate professors, and one is a full professor, all experienced in teaching mathematics and the methodics of mathematics. The average length of working experience at a school or university is around 12 years. Six participants have five or fewer years of working experience, six of them have more than five and less than 18 years of experience, and three have more than 18 years of working experience.

The bottom-up thematic analysis, or the inductive method as it is also called, was used for the data analyses (Fereday & Muir-Cochrane, 2006).

Results

Teachers’ conceptions of mathematical literacy The analysis of the participants’ responses to the first set of questions showed that primary and secondary school mathematics teachers define mathematical literacy as the ability to correctly use mathematical symbolism, as knowledge about the basic mathematical concepts and their application in everyday situations. One-fourth of the interviewed teachers (4 out of 12) believe that mathematical literacy includes the correct use of mathematical symbolism, and two of them believe that literacy is exclusively the use of the symbolism:

In. 5: “ [to be mathematically literate] means that the students can use mathe-matical symbols appropriately and correctly and to understand their meaning … This should be considered as mathematical knowledge.”

Many teachers (8 out of 12) believe that mathematical literacy is knowing the basic mathematical concepts and basic mathematical operations. Three believe that literacy is exclusively knowledge about the concepts:

In. 14: “to be mathematically literate means to know the basic mathematical concepts, what is a cone, line segment, point, line… to be informed and know these concepts.”

Five teachers believe that mathematical literacy includes the application of knowledge in everyday situations, and one of them is exclusive about this:

In.1: “To be mathematically literate is to be literate for life. To manage and use knowledge in everyday life… percentages, house painting, markets, banks… plus higher levels in some professions.”

Only two teachers expressed opinion that, a special way of thinking has its role in mathematical literacy:

In. 10: “… Generally, the way of thinking and concluding, approaching a problem, data collecting, but not solving.”

Similarly to the latter example, university mathematics professors agree that mathematical literacy includes special ways of thinking, but they explain it differently. One line of thinking is that literacy includes only the interpretation of mathematical problems; the other is that it should include problem-solving. Lastly, literacy is considered as an application of mathematical thinking in general.

In. 9: “… to ‘read’ mathematical problems and understand them… to interpret, but not to solve. During education, mathematical literacy is gained by exposing the students to problems and situations, but this does not mean that children can also solve these problems.” In. 7: “There is basic mathematical knowledge that should remain after we finish Serbian education. The influence that mathematics has on the overall development of a person is far more important than the use of mathematics in everyday life. For me, being literate means adopting the mathematical and analytical way of thinking and problem-solving.”

Participants’ views on mathematical assessment and teaching materials During the interview, most primary and secondary school teachers expressed the opinion that, for the lowest passing grade (grade two), the students should recognize the basic concepts and conduct basic procedures on them (eight out of 12). Two teachers require the students to solve tasks and not to merely recognize basic concepts. For the middle grade (grade three), teachers require more complex procedures and calculations (six out of 12 teachers) and the application of knowledge (three teachers). For the highest grades (grades four and five) teachers require interconnection (two teachers) and application of knowledge (two teachers), work on “more complex textual tasks” (In. 5), and “unknown and unstandardized tasks” (In. 1). Five teachers emphasized that, for the highest grades, the students need “to think.”

University professors believe that it is the way of students’ thinking that plays a critical role in evaluation, not the volume of reproduced content.

In. 7: “For the grade two [the lowest passing grade], one is expected only to recognize [mathematical facts]. For other three grades [three, four or five], one has to reproduce, but there are only slight differences, who is more skillful at it… In my opinion, this should be expected for grade three, while for four or five, one has to manage a new, unstandardized situation, to apply (knowledge) on the new level.”

About PISA and the final exam With regards to PISA, six primary and secondary school teachers expressed a positive attitude towards its tasks and manner of assessment. They find the tasks “interesting,” “practical,” and that “they make you think a little harder.” Other six teachers are familiar with this study from the media, but they never saw PISA tasks. Some of the teachers who have a positive attitude towards PISA tasks expressed the concern that the students in Serbia do not have the opportunity to become acquainted with similar content (two teachers) and to learn how to think logically and draw conclusions (two teachers), which is needed for solving these types of tasks.

All three university professors expressed a reserved view of PISA. They find this study important, but they emphasize that it does not assess pure mathematical skills and essential mathematical connections and, because of that, it should not be considered as the only measure of mathematical knowledge.

In. 9: “It [PISA study] has several deficiencies. In my opinion, the significance of PISA testing is overemphasized. It gives some overview, it is important, but it should not be overemphasized. The objectivity of the approach is especially important, and it is not sufficiently present in this test. Secondly, the tasks are very contextualized, which surpasses Serbian children’s knowledge. Thirdly, the approach is very pragmatic – PISA does not test the understanding of essential mathematical connections. I am against a pragmatic approach to mathematics, as well as to reading. We do not read just to understand a text. There are higher levels. The problem with the practical approach is important in the sense of what should be measured and encouraged.” In. 7: “Overall, I welcome that way of testing on large samples and to make comparisons with other educational systems. But, I don’t think that it is the only measure of efficiency, just as Mathematical Olympiad (in which we are very successful) are not. None of this represents a whole.” In. 12: “PISA is useful testing, but it is often misinterpreted as something that exclusively indicates what was learned in school. It is constructed to evaluate functional knowledge and not the ability of the student for further learning of mathematics. It measures the application of knowledge, and that is useful, but it is not the only thing that children should know. That is dangerous. That way, we send the message to children that they do not need further education, that after school, we are going straight into “life” and that we have learned enough.”

When it comes to the final exam, many participants (primary and secondary school teachers and university professors) expressed a positive attitude towards it. They found the tasks adequate and generally good (eight out of 15 participants). They considered the final exam suitable for measuring mathematical literacy (eight participants) or at least to some extent suitable (five participants). The greatest concern was the lack of evaluating the solving process in the tasks (nine participants). Additionally, some participants expressed the opinion that it is absurd to give identical tasks (on the exam) to those which the students have in their preparation textbooks (four of them). It is a kind of “anti-mathematics” as it was pointed out in one of the interviews. Five participants thought that the tasks are too basic and straightforward; three of them pointed to the lack of logical tasks and two to the lack of tasks with the application of knowledge.

About the pairs of mathematical tasks

There are several common answers to the questions posed regarding the three pairs of tasks presented to the participant during the interview (given by both the mathematics’ teachers and university professors). In connection to all of the presented tasks, the participants emphasized that PISA tasks would be more interesting to the students, and most of them would like to see these kinds of tasks in the textbooks.

There was also a common ground for the main reason for the low achievement on PISA tasks. The main reason seems to lie in the unfamiliarity of the situations used in PISA tasks and the students’ fear of unknown types of tasks. By analyzing one of the pairs of tasks, the participants outlined potential problems in solving PISA tasks. Nine participants saw a problem in the way PISA tasks were formulated, in the selection of the data needed to solve the problem (five participants), and in the students’ reading literacy (six participants).

For example, in the presented pair of tasks (Figure 2), the participants found PISA tasks to be complicated and expected that a very small number of students would be able to solve them in comparison to the final exam tasks (eight participants). On the other hand, four participants believed that students should be more successful in solving PISA tasks because they do not require knowledge about formulas, graphs, etc. The students should only read the data from the graph, and the answer could be assumed from the offered answers. The rest of the interviewees considered PISA tasks to be more challenging but not drastically.

The participants description of the tasks from the final exam (Figure 2) was that it is “classical,” “standard,” and “well-known” (seven participants), and some of them described it as “stereotypical” and “dull” (four participants). On the other hand, PISA tasks are perceived as requiring understanding and thinking (four participants).

Discussion

Looking at the literature, we can observe that there are various nonconfronting descriptions of literacy (Group, 2009; OECD, 2005; Rajović & Radulović, 2007), but it is not easy to find a consensus regarding the definition of mathematical literacy (Jablonka, 2003). However, despite this, there is voluminous research that emphasizes the importance of the mathematical modeling process and contextualized word problems (e.g.,

Blum & Leiss, 2007; Verschaffel et al., 2000; Verschaffel et al., 2014), which engage important thinking processes. Therefore, the concept of literacy, greatly operationalized through PISA tasks, is important, and teachers should, therefore, be familiar with it.

However, similarly to the theoretical dispute, in our study, participants had different views on literacy. The problem does not lie in contradictory viewpoints, but rather in different participants highlighting different aspects of this concept. However, this is often the case with concepts – practitioners frequently have a different interpretation (and misinterpretation) of concepts based on their experience and general beliefs.

Most of the participants perceive mathematical literacy as knowledge about basic mathematical concepts and basic mathematical operations. This fits almost perfectly into the first two aspects of mathematical literacy conception described by Kilpatrick (2001) (conceptual understanding and procedural fluency) but not the others, which are more closely connected to the OECD concept of literacy ( strategic competency, adaptive reasoning, and productive disposition ). Additionally, some participants perceive literacy as the use of mathematical symbolism and make the difference between mathematical literacy, mathematical knowledge, and problem-solving. According to their point of view, mathematical literacy includes the use of symbols and their understanding (In. 7: “to read and understand what was read…”), but it does not include problem-solving, and it is a narrower concept than mathematical knowledge (In. 5). This definition is incompatible with the OECD definition of literacy, which assumes representing something outside the learned system of symbols. It is not surprising that teachers have different views on mathematical literacy compared to the OECD interpretation. As we stated in the introduction, they have different views on mathematics that only recently began to change by slowly introducing the concept of literacy as presented in the OECD conception. The fact that many teachers are not familiar with the PISA study indicates that this concept is yet to be presented to Serbian practitioners. This implies some recommendations for the initial education of mathematics teachers in Serbia, which should include teaching these concepts to the students.

Almost half of the primary and secondary school mathematics teachers (five out of 12) assume that mathematical literacy is the application of mathematics in everyday situations. All of them emphasized only basic life routines; they

gave coping with problems in markets, calculation with percentages on sales, etc. as examples (e.g., In. 1). Teachers did not mention any of the problems that could be found in the PISA framework (such as energy efficiency, choice of the new car depending on its characteristics, etc.). We consider this as a confirmation of the attitude that mathematical literacy is a concept that varies depending on the cultural setting of individuals (Jablonka, 2003) and what is considered as the main problem-solving situation that people usually encounter in society.

Although the sample of primary and secondary school teachers and university professors was uneven in this study, it is noticeable that university professors highlighted the special ways of thinking when defining literacy, which was not that common in primary and secondary school teachers’ definitions. Also, it is noticeable that in their definitions, university professors highlighted the interpretation and problem-solving aspects, while the Blum and Leiss’s model (2007), as well as the OECD model, included several aspects – mathematical thinking, justification, communication of the results, etc. This might be connected to the common understanding that literacy is the basic level of competency (similar to the understanding that being literate means being able to read and write) and that mastering all aspects and processes in mathematics is connected with expertise.

To sum up the results connected to the second research question, we can state that primary and secondary school teachers from our study have different views on assessment compared to university professors. Teachers think that the average student (the one assessed with grades two or three) should recognize the basic concepts and conduct basic or more complex procedures (for the grade three). Students with the highest grades (four or five) should be engaged with an interconnection of knowledge, applications, textual tasks with higher complexity, unknown tasks, etc. If we take the premise that the education system should provide (mathematically) literate students, then Serbian teachers consider the recognition of the basic concepts and conducting basic or more complex procedures as obligatory part of mathematical literacy, which was often mentioned in the definition of mathematical literacy at the beginning of the interview. Teachers’ views on the final exam are also in alignment with their conception of mathematical literacy. Most teachers consider that the final exam measures mathematical literacy. Also, they find that the tasks are good, well-chosen, and cover all mathematical fields. On the other hand, some teachers consider the tasks to

be too easy or trivial, and some emphasized the lack of logical or applicational tasks.

Half of the primary and secondary school teachers, who participated in this research, are familiar with PISA tasks and expressed an overall positive opinion about the PISA test, while others never came across PISA tasks. Taking a look at PISA tasks presented in this study, almost all participants agreed that PISA tasks would be more interesting to the students than traditional tasks and that they would like to see this kind of tasks in the textbooks. Low achievement on PISA tasks is often seen as a consequence of unfamiliarity of the situations present in the tasks, students’ fear of unknown types of tasks, ways that PISA tasks are formulated, the selection of the data needed to solve a problem, students’ reading illiteracy, etc. This is in accordance with the previously discussed stance that PISA situations are not ‘real life’ situations for the students in Serbia. From the teachers’ point of view, if PISA tasks and PISA situations became known to the students, the results on the PISA test would be better. This raises a series of philosophical questions. Should the goal be the success on the test? Is this kind of mathematical framework right for reaching mathematical literacy? It is clear that someone who has mastered mathematics as science could easily solve tasks in PISA or any other framework. The question is: are PISA-like tasks the way to master mathematics and be able to use it in any other framework? University professors in our research are reserved about the last question. “PISA does not test essential connections that exist in mathematics,” as In. 9 stated. The professors emphasize that it is not the only measure of efficiency in mathematics (In. 7) and it does not measure “the ability of the student for further learning of mathematics” (In.12).

Participants from the sample showed interest in including PISA-like tasks in the teaching process. Most of them find these tasks suitable for fostering mathematical thinking and motivating students. Many researchers refer to contextualized word problems and the modeling process as a good means for the development of meaning in mathematics and fostering some of the important mathematical processes (Blum & Boromeo Feri, 2009; Blum & Leiss, 2007; De Corte et al., 2000; Verschaffel et al., 2000; Verschaffel et al., 2014). We could say that the Serbian mathematics teachers’ community is ready to accept and apply these aspects of mathematics education.

Conclusion and implications

Our study confirms that mathematics teachers, as well as the research community, do not have a shared understanding of mathematical literacy. Four different conceptions of mathematical literacy were found in this study: 1) mathematical literacy as knowledge about basic mathematical concepts; 2) mathematical literacy as the correct use of mathematical symbolism; 3) mathematical literacy as the use of mathematics in everyday situations; 4) mathematical literacy as the assimilation of a specific way of thinking. The first two concepts were dominant in the teachers’ definitions, while only a few mentioned the special way of thinking. This is not in accordance with the OECD definition and some other definitions that could be found in the research literature (e.g., Kilpatrick, 2001).

Many teachers recognized the application of mathematics in everyday situations as an important part of mathematical literacy. Their description of this application gives the opportunity for illuminating this concept from the cultural point of view (Jablonka, 2003). Teachers’ examples used for describing the application of mathematics in everyday situations are far more different from the contexts used in the PISA framework.

In contrast to the primary and secondary school teachers, university professors highlighted that mathematical literacy should include some of the ways of thinking: interpretation, analytical thinking, and problem solving. The communication of reasoning and results, validation, and exposure were not present in their definitions (but these aspects are well recognized in literature).

Teachers’ and professors’ views on the assessment practice, the final exam, the PISA test, and the diverse types of mathematical tasks confirmed that their practice is in accordance with their conception of literacy. On the other hand, it revealed some of their significant beliefs and intentions for future practice. Teachers showed interest in including PISA-like tasks in their practice and recognized the importance of the modeling process. This kind of tasks is recognized only as an addition to their existing practice. This was especially emphasized by university professors, who highlighted that, although this framework is useful for one part of mathematical education, some other approaches to teaching mathematics are important as well.

The fact that a higher level of reading ability is necessary for solving PISA tasks as participants noticed shows us that these tasks do not measure only

the attainment of mathematical concepts but rather a very complex competency that consists of different cross-curricular skills and knowledge. This is not surprising since it is in line with the OECD concept of literacy but has implications for the teaching practice. Teaching such complex, crosscurricular competencies takes a cross-curricular approach and formative assessment that will indicate a specific kind of difficulties that the students encounter (e.g., low reading ability, unfamiliarity of presented situations etc.).

Overall, the results of this study showed that although we do not have a shared understanding of the concept of mathematical literacy, the key skills and knowledge making up mathematical literacy, as defined by OECD, are recognized as important. PISA-like tasks are recognized as a good teaching material that can promote the development of the students’ mathematical competencies and motivation for learning mathematics. However, by doing so, the teachers must keep in mind that the success on these tasks depends on the level of the students’ other competencies and previous experience. Overall, the participants also question whether the PISA point of view is the only one and highlight that the practical approach is not the only relevant approach but that mathematics should instead be thought of as a formal discipline. This is consistent with the general notion derived from the sociocultural approach, according to which the development of the system of scientific concepts should lean on spontaneous, everyday concepts, but it should also contribute to the understanding and further development of everyday concepts.

References

Blum, W., & Boromeo Feri, R. (2009). Mathematical Modelling: Can It Be Taught and Learnt? Journal of Mathematical Modelling and Application, 1 (1), 45-58. Blum, W., & Leiss, D. (2007). How do students and teachers deal with modelling problems. In C. Haines, P. Galbraith, W. Blum, & S. Khan (Eds.), Mathematical modelling: Education, engineering, and economic s (pp. 222-231). Chichester: Horwood. De Corte, E., Verschaffel, L., & Greer, B. (2000). Connecting mathematics problem solving to the real world. Proceedings of the International Conference on Mathematics Education into the 21st Century: Mathematics for living (pp. 6673). Amman, Jordan: The National Center for Human Resource Development. Fereday, J., & Muir-Cochrane, E. (2006). Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development. International journal of qualitative methods, 5 (1), 80-92.

Group, P. L. E. (2009). PIAAC Literacy: A Conceptual Framework. Retrieved on 15 May 2018 from https://doi.org/10.1787/220348414075 Jablonka, E. (2003). Mathematical Literacy. In A. Bishop, M.A. Clements, C. Keitel, J. Kilpatrick & F.K.S. Leung (Eds.), Second international handbook of mathematics education (pp. 75-102). Dordrecht: Kluwer Academic Publishers. Kilpatrick, J. (2001). Understanding Mathematical Literacy: The Contribution of Research. Educational Studies in Mathematics, 47 (1), 101-116. OECD (2005). The Definition and Selection of Key Competencies: Executive Summary. Retrieved on 14 January 2018 from: https://www.oecd.org/pisa/35070367.pdf OECD. (2000). Measuring Student Knowledge and Skills. Retrieved on 15 May 2018 from https://doi.org/10.1787/9789264181564-en OECD. (2006). Assessing Scientific, Reading and Mathematical Literacy. Retrieved on 15 May 2018 from https://doi.org/10.1787/9789264026407-en OECD. (2013). PISA 2012 Assessment and Analytical Framework. Mathematics, Reading, Science, Problem Solving and Financial Literacy. Retrieved on 15 May 2018 from https://doi.org/10.1787/9789264190511-7-en OECD. (2016). What does low proficiency in literacy really mean? Retrieved on 15 May 2018 from https://doi.org/10.1787/5jm0v427jl9p-en Pavlović-Babić, D., & Baucal, A. (2013). Podrži me, inspiriši me: Prvi rezultati PISA 2012 u Srbiji. Beograd: Institut za psihologiju i Centar za primenjenu psihologiju. Philipp, R. A. (2007). Mathematics teachers’ beliefs and affect. In F. K. Lester, Jr. (Ed.), Second handbook of research on mathematics teaching and learning , (pp. 257315). Charlotte, NC: Information Age. Radišić, J., Videnović, M., & Baucal, A. (2015). Math Anxiety – Contributing School and Individual Level Factors. European Journal of Psychology of Education, 30 (1), 1-20. Radišić, J., Baucal, A., & Videnović, M. (2014). Unfolding the assessment process in a whole class mathematics setting. Psihološka istraživanja, 17 (2), 137-158. Rajovic, V., & Radulovic, L. (2007). Kako nastavnici opazaju svoje inicijalno obrazovanje: na koji nacin su sticali znanja i razvijali kompetencije. Nastava i vaspitanje, 51, 413–434. Regulation on General Standards of Achievement – Educational Standards for the End of Compulsory Education 2010 Thompson, A. G. (1992). Teachers’ beliefs and conceptions: A synthesis of the research. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 127–146). New York: Macmillan. Verschaffel, L., Greer, B., & de Corte, E. (2000). Making Sense of Word Problems. Lisse: Swets & Zeitlinger. Verschaffel, L., Depaepe, F., & Van Dooren, W. (2014). Word problems in mathematics education. In S. Lerman (Ed.), Encyclopedia of mathematics education (pp. 641–645). Dordrecht, the Netherlands: Springer.