Examining instructional change at scale using data from diagnostic assessments built on learning trajectories

This study investigated the process of instructional change required to translate data on student progress along learning trajectories (LTs) into relevant instructional modifications. Researchers conducted a professional development session on ratio LTs, which included analyzing 3 years of district-level data from Math-Mapper 6–8, a digital LT-based diagnostic assessment application, with fifteen 6th and 7th grade teachers. Teachers subsequently conducted a lesson study to enact what they had learned, allowing researchers to study how teachers used data on student progress along ratio equivalence LTs to design, implement, and evaluate the lesson study. Researchers applied a framework for LT-based data-driven decision making to analyze video data of the lesson study activities. Teachers successfully scanned data reports to pinpoint the LT levels at which to target modified instruction. In one instance, they focused too narrowly on a single item resulting in excessive lesson time on tasks on graph literacy external to the LT. In the other, their data interpretation was overly general and resulted in the design and implementation of a sequence of tasks that reversed the order implied in the LT and relied on the use of more sophisticated strategies from subsequent LTs. Results suggest a need for more data interpretation skills, a deep understanding of the learning theory underpinning LTs, and more precision in teacher discourse around LTs.


Introduction
Learning trajectories (LTs) are promising candidates as learning theories to underlie diagnostic, formative classroom assessment (Shepard et al., 2018). By assessing students' progress along LTs during instruction, teachers systematically obtain evidence with which to modify their instruction to meet individual and class needs and improve learning outcomes. Math-Mapper 6-8 (MM), a diagnostic assessment software application that supports teachers to assess student progress along LTs for all middle grades topics, provides an opportunity to study the process of instructional change associated with the use of LT-based classroom assessment.
Viability of LT-based classroom assessment for effective instructional and learning improvement will depend heavily on how teachers interpret and participate with data as a resource in the classroom (Remillard, 2005). The means by which teachers participate in the use of data is mediated by the MM software, and hence, according to the theoretical ideas of an instrumental approach, a bidirectional process of instrumentation and instrumentalization occurs by which the design of the software influences teachers' practices and teachers' practices shape the use of the software (Rabardel, 1995). Gueudet and Trouche (2009) propose a documentation approach as a means to study the genesis of practices associated with digital resource use. In our study, these processes have been consistently examined over the course of the software development by studying teachers' use patterns and identifying three factors differentiating more and less effective use of data. Teachers who effectively use data were able to: (1) successfully, supportively shift from a summative to a formative orientation, (2) exhibit a stronger understanding of the concept and content of the LT, and (3) identify and enact specific actions warranted by data . These results revealed a need for a framework to guide interpretation and translation of classroom assessment data into instructional actions and provide insight into 1 3 "how and to what degree do teachers' select, use, and interpret data to propose instructional modifications to improve student progress along LTs?".
We propose a LT-DDDM (LT-based, data-driven decision making) framework and use it as a lens to examine how a group of teachers interpreted and applied data on their students' performance on a ratio equivalence LT.

Theoretical foundations
We theorize that if teachers were more knowledgeable and informed about students' progress along LTs, they would be more successful in fostering learner-centered instruction; this in turn would produce stronger student learning outcomes. We see the use of diagnostic, formative assessment of progress on LTs as the means to achieve these connections.

Learning trajectories
LTs are descriptions of levels of thinking learners are likely to display as they move from naive ideas to more sophisticated, conceptual understanding of a target concept (Battista, 2011;Clements & Sarama 2014;Confrey, 2019;Van Den Heuvel-Panhuizen, 2003). Each LT exhibits five essential properties: 1. The LT has a clearly defined and worthy target or goal; 2. All level topics are important, though some may be more important than others; 3. Levels are sequenced in order of sophistication; 4. Each level is cognitively related to the next to engender movement and support connections; and 5. Multiple learner paths can be supported.
LTs are synthesized from empirical studies (clinical interviews, teaching experiments, and design studies) of children engaging with challenging, developmentally appropriate, instructional tasks. Researchers observe how students draw on prior knowledge, experience, and cultural assets to interpret the task, propose, and incrementally work towards solutions over time (Confrey, 2019). The developmental process is viewed as genetic-epistemological (Piaget, 1970) or guided reinvention (Freudenthal, 1987). LT levels comprise a variety of epistemological objects: representations, strategies, cases, forms of reasoning, inventions, and partial conceptions arranged in levels of sophistication. LTs do not constitute a stage theory; instead, they offer descriptions of thinking that is likely to emerge in classrooms when rich learning opportunities are made available (Lehrer & Schauble, 2015). LTs are written at a variety of grain-sizes, depending on whether they are used to frame longer curricular sequences (Wilmot et al., 2011) or guide instruction on a daily basis (Simon, 1995).
The ratio equivalence LT used in the study illustrates the richness of epistemological objects (Confrey, 2019) and constructivist sequencing that comprise research-based LTs (Fig. 1). The LT draws initially on a learner's intuitive understanding of ratio as a relationship between two quantities that remains the same when both quantities are multiplied or divided by the same factor (e.g., a recipe) (Lesh et al., 1988), and addresses the additive misconception (Hart, 1988). After making more (Level 1, or L1), students make less (L2, L3). By L4, students recognize that they can generate and organize equivalent values into tables. By sequencing table values, they see patterns connected to making more and less, extend the tables to include (0,0), and explore patterns of change showing how x and y covary. The target of the LT, L5, involves students recognizing how patterns in co-varying quantities translate graphically into straightness. Developing a correspondence relationship between x and f(x) = kx is Articulating likely levels of student reasoning, LTs afford teachers a tool to strengthen their own mathematical knowledge for teaching (Ball et al., 2008). Instead of treating an absence of knowledge as a deficit, they support insight into specific ideas that lead to more sophisticated conceptual understanding. Armed with knowledge of students' progress on LTs, teachers can be encouraged to strengthen their focus on students-their ideas, approaches, and reasoning. Thus LTs can be viewed as a means to strengthen learner-centered instruction (Confrey et al., 2017;O'Shea & Leavy, 2013) (Fig. 2).

Classroom formative assessment
One way to deploy LTs to increase learner-centered instruction is to provide teachers data on student progress along LTs during instruction, as a basis for modifying instruction. This approach implies a formative use of assessment. In fact, new forms of formative assessment-more systematic, comprehensive, and based on specific theories of learning (Shepard et al., 2018)-are emerging under the descriptor "classroom assessment" (Pellegrino et al., 2001).
Using such a learning theory-based, systematic approach to formative assessment adds a dimension to classroom practice that has so far eluded practitioners: how to use systematically measured, comprehensive data from measures of student learning. Research on practitioners' use of data indicates that "Achieving the vision of data-use advocateswhere teachers' regularly respond to data with changes in their instruction-is a complicated affair" (Farrell & Marsh, 2016, p. 407). Although reviews of digital curricular resources (DCRs) recognize that networked classroom technologies scaffold an increased use of formative approaches (Pepin et al., 2017), between generation of, and response to, data lies the challenge of ensuring that data interpretations value learners' thinking. Teachers can achieve changes in their practices when they: (1) actively make conjectures about data in relation to learning, (2) modify and strengthen instruction based on their conjectures (Hamilton et al., 2009), and (3) invite students to actively participate in data review processes, thus prioritising student agency and selfregulation (Heritage, 2007). Teachers can also deepen their own thinking about the mathematical content when working in professional learning communities (PLCs) to explain the reasoning underlying student errors (Brodie 2014).
Math Mapper 6-8 (MM) is the classroom-based, formative assessment tool used in the study. MM provides systematic feedback on students' progress along 62 constructs for middle school math. LTs were developed by synthesizing empirical research on learning. A typical use model is that teachers: (1) conduct initial instruction, (2) administer a relevant MM assessment, (3) review data, and, (4) in response, modify their instruction to support deeper student understanding of an LT's target concept.
Teachers can administer digital assessments, each comprising 8-12 conceptually rich items with distractors to diagnose student understanding, misconceptions, or errors. Student reports allow students to review their overall percent correct, item-by-item, level-by-level performance, and can revise or reveal answers to missed questions. From their reports, they can also access MM's practice features for levels they found most difficult. Teacher reports (heatmaps; 3) display detailed results for all students in a class. Teachers working in PLCs can also use their collective data to justify long-term curricular changes.

Framework for LT-based, data-driven decision making
A framework for data-driven decision making was necessary in order to study how teachers' select, use, and interpret data to improve student progress along LTs. We revised Hamilton et al. (2009)′s data-use cycle to apply it to an LT-based diagnostic formative assessment system, resulting in a four-component framework (Fig. 3) focused around strengthening learner-centered instruction and the intended outcome of measurable improvement in learning. Specific LT-based actions are provided for each component.
To begin, teachers scan data for each LT, presented in heat maps (Fig. 4). Each cell represents a student's data from one task at a given level, color-coded from orange (incorrect) to blue (correct). Levels are ordered vertically and students are ordered from weakest to strongest performing on that LT. Heatmaps display the overall strength of the relationship between levels and ordered student performance. Next, based on where the blue or light blue transitions to orange, teachers identify the upper boundary of class proficiency on levels and identify segments of struggling or proficient learners.
In the second component, teachers interpret data by contextualizing them in relation to the class and past curricular and instructional experiences. They make specific conjectures of how to improve learning outcomes. Conjectures use the following format:, "based on data from < topic > LT showing evidence of < student learning > in [designated levels, items, or options], we conjecture that < some/all > students will improve learning if they develop more proficiency in < idea > ." To develop their conjectures, teachers connect relevant data to the map's hierarchical structure. They examine individual items, including students' responses or choices of distractors. They situate items within levels, which in turn are situated in sequences of levels of increasing sophistication. The hierarchical analysis strengthens ties between the conjectures and underlying ideas in the LTs.
In the third component, teachers plan and implement instructional changes and test their conjectures. We observed teachers using four general organizational strategies: 1. Conduct whole class review of weaker levels; 2. Form groups based on LT evidence, to revise and resubmit incorrect items 3. Use data to advise students on which LT levels to practice; and 4. Select and try new tasks based on conjectures and related LT evidence.
We postulate that each strategy can be conducted in a more or less learner-centered way depending on the strength and specificity of conjectures. Learner-centered instruction also depends on the degree to which teachers explicitly situate these activities as formative experiences.
The fourth component involves reassessing to see if one's conjectures and instructional modifications were accurate and successful. MM's psychometrically equated retests are primary resources for this. This component also includes methods of reviewing the overall episode as a negotiated interactive enactment of a hypothetical LT (Simon, 1995) and a collective story about learning.

The research study
We sought to learn how teachers use data from MM to make instructional modifications. A collaborating district's 6th and 7th grade teachers had participated in an LT-based ratio reasoning PD session (including a review of the district's data) conducted by the authors. Afterwards, the district supervisor asked teachers to conduct a lesson study focused on one of the workshop's LTs. A subgroup of 7th grade teachers selected ratio equivalence, volunteered to plan the lesson, and share their materials and approach with colleagues. One 7th grade teacher taught the lesson, and the rest analyzed and discussed the instruction. The research question was: During the planning, conduct, and analysis of a 7th grade lesson study on ratio equivalence, to what degree and how do teachers' select, use, and interpret data to propose instructional modifications to improve student progress along LTs?

Context
The study was conducted with teachers and their supervisor in a high-performing K-8 district in New Jersey. A total of 1394 students are enrolled in the district's two middle schools (77% White, 9% Hispanic, 4% African American, 8% Asian). 15 teachers participated in the PD session and lesson study. The lesson, taught by Sam, an experienced MM user, took place in his 7th-grade regular (not accelerated) class.

Professional development session
The 6 h PD session focused on five out of MM's six ratio LTs and corresponding district-level data ( Fig. 9 and 10). The ratio equivalence LT was discussed for an hour with emphasis on defining a new kind of equivalence, strategies to address the additive misconception, patterns in tables of values, and graphing ratios on the plane. We reproduce two PD artifacts later incorporated in the lesson study: (1) item aligned to L5 of the ratio equivalence LT, which influenced the teachers' instructional modification design (Fig. 5). (2) an explanation and diagram of the instructor's approach connecting L4 and L5. She warned against overreliance on plotting points from tables and inductively concluding they formed straight lines through (0,0), instead emphasizing how the table's structure (from making more or less) or notating covariation could be used to explore straightness (Fig. 6).

Data sources
Lesson studies comprise four episodes: planning meeting, pre-lesson meeting, lesson implementation, and debrief. Their associated purposes are to: agree on a research theme, plan the research lesson, collect data, and consolidate subject-matter take-aways, student thinking, and instruction (Lewis & Hurd, 2011).
Data for the study consisted of video of a 30-min planning meeting, 11-min pre-lesson meeting, 1-hour lesson implementation, and 1-hour debrief. Following initial analysis of the data, we interviewed the lead teacher to learn more about planning decisions and implementation. Before and after the lesson, participating students took a pre-and posttest ( Fig. 11) on ratio equivalence.

Data analysis
We recognized that the four components of the LT-DDDM framework corresponded closely to the episodes in the lesson study. Component 1, "gather and scan data", was most evident in the discussions of data from the workshop and planning session; component 2, "interpret data on how to improve learning", was observed during the pre-lesson meeting as teachers explained their goals for the class; component 3, "modify instruction and test conjectures" applied to the lesson implementation, and component 4, "reassess and review" corresponded with the debrief and follow-up interview. This mapping allowed us to analyze the empirical data by lesson study episode and connect it back to the data-driven decision framework.
On our first pass through the video data, using constant comparative analysis of transcripts (Boeije, 2002), we identified eight cross-cutting themes that related to LT meaning, data, and instructional planning. We categorized the eight themes in relation to the components of the LT-DDDM framework (Table 1). Using the framework, we analyzed the data by lesson-study episode, considering how the data use influenced the design, implementation, and evaluation of the lesson study.
We transcribed the lessons, mapped out the logic of the lesson study, and related framework components to the activities. We analyzed the lesson tasks in relation to the structure of the LT and the teachers' expressed goals and expectations of students. Significant attention was paid to teachers' discussions of content and pedagogy. During the lesson, we focused on: (1) the extent and type of teacherstudent interactions, (2) examples of the lead teacher's language as he presented tasks and asked questions, and (3)

Results
Highlighting exchanges related to the themes, we present results in the form of the lesson study episodes as they unfolded in time. Then we use the LT-DDDM framework to analyze the results, seeking to determine how the teachers' approach to lesson study represented a cycle of inquiry involving both data-driven decision making and appropriate referral to the LT levels.

Planning session
The planning team, consisting of Sam, two 7th-grade coplanning teachers, and the supervisor, met for 30 min. Subsequently, Sam and two other teachers developed the curriculum materials. Sam proposed using "ratio equivalence" as the lesson study topic and read the levels verbatim: ...I do like that one. So, the levels being...you make ratios more by multiplying both quantities, you make less by dividing both quantities, by two, one or more times. They create equivalent ratios by making less by dividing both quantities by a factor.
He also read L4 and L5 verbatim. The choice of a 6th grade topic for the lesson study was driven by the district-level data which showed that the non-advanced 7th grade classes lacked proficiency in the idea. Selecting ratio equivalence, the planning team discussed two possibilities for context: paint or lemonade.
The group recalled that the district data showed weak performance at L5 and that the supervisor had challenged the teachers to improve student performance at the upper levels, "So the goal would be to talk about those top two levels-four, five." She and Sam briefly reviewed the meaning of the level and how to approach it: Sam : So, I think we could eventually connect this [L5] through plotting their points. In terms of lemons and sugar [gestures axes of the plane with arms]. And even like having different groups plot different values and seeing, do they all fall on the same line? And how does that come about? And then realizing what are the similarities and differences in your graphs. Component 1: Gather and scan data Primary data source: workshop and planning meetings Descriptions of LT-levels structure, and connections between levels Teachers' discussion of where to focus the lesson study in relation to the structure of the LT and their lack of attention to connections between consecutive levels Articulation of focus and rationale for topic selection in lesson study (including supporting data) Teachers' analysis of MM data from the ribbon diagrams and subsequent of L5 as the focus of the lesson study Component 2: Interpret data on how to improve learning Primary data source: pre-lesson meeting Reference to student characteristic and needs Teachers' expressed concerns for balancing the needs of students who have yet to gain proficiency in the LT with those who are proficient, and encouraging students to participate in class discussions References to curricular resources (task selection, goals, and expectations) Teacher explanations of lesson study tasks without articulation of expected student behaviors Component 3: Modify instruction and test conjectures Primary data source: lesson implementation Teachers' presentation of task in terms of language used, questions asked, proportion of time spent on classroom interactions Despite the supervisor's invitation to discuss the meaning of the line, the discussion moved to pedagogical brainstorming about L5, without discussing L5′s relation to L4. Throughout the planning, teachers tended to shift to content-free discussion about pedagogy and to use imprecise or incomplete language whenever specific content was invoked.
Sam then shared an internal debate about how to approach the first four 6th-grade levels with 7th-grade students saying, "This is a 6th-grade topic that I know that they talk about in 6th grade. I do get nervous that there will be some that will come in and be like, 'I know this already.'" Only one teacher specifically discussed the content and student mastery of the early levels, indicating that it is hard to know if students understand even the lowest level on their own: "…somebody might instinctively say 2 plus 3 but once somebody says double it, I think the others are going to go 'yeah!'" She seemed to be suggesting that many will know that equivalence is preserved if they double both quantities. However, she surmised that pressure from other students' use of the term "doubling" will quickly dominate and suppress the additive misconception, presumably without full attention to the underlying reasoning. Nonetheless, teachers saw little need to discuss the lower levels 1 beyond ensuring that the students knew the procedure of doubling/multiplying by a factor.

Pre-lesson meeting and curriculum analysis
The second episode of the lesson study included a deputy superintendent, the supervisor, a support teacher, four 6thand four 7th-grade teachers.
The 7th-grade co-planning teachers reviewed final tasks for three stations. They explained that all students would be asked to solve the pink frosting problem, representing L5. Then they would rotate in groups through the stations. 2 A co-planning teacher summarized a goal of station 1 (Fig. 7) as: The key is to explain how you know, whether the kids know from the table, great, but how do you know from the graph, whether or not? And if you know from the graph, how do you know from the table? So relating the two to one another and being able to write out why that is. An abbreviated statement of the goal ended with, "from the table…" and "from the graph…". We assumed it would conclude with "whether it represents ratio equivalence." Station 1 consisted of four problems, each checking for ratio equivalence within a table and graph. Problems provided, in order, both a table and a graph, a table only, then a recipe (requesting a table and a graph). Including a table and graph in problem 1 and 2 aligns them with L5, whereas problem 3 seems to scaffold the discovery of L5, and completing the table for an even-valued recipe on problem 4, aligns with L1 and L2. It was not clarified how students were expected to solve the problems or explain their reasoning.
The teachers devoted two-thirds of the pre-lesson meeting time discussing what variables (boys, girls) students would place on the axes.
Station 2, with the goal of matching graphs to tables, presented 3 tables and 4 graphs showing relationships between "cups of sugar water" and "lemons" (Fig. 8). Two tables had two matching graphs each (with reversed axes).
Teachers wanted to focus students' attention on axis labels and careful interpretation of the coordinate points: Co-planning Teacher: The overarching idea here is understanding not just proportionality but understanding what it looks like on a graph and could you switch it up and still be right? Does it still show that ratio?
Does it still show that relationship? And is it okay, why is it ok?...
During this pre-lesson meeting, the co-planning teachers did not explain how the station 2 task reflected an interpretation of the LT and related data.
Station 3 required the students to reflect on the frosting problem from the beginning of the class and consider revising their answers: Co-planning Teacher: So, you'll see that continuous question kind of through the stations too, because that next station that we just got over, talked about sugar water and the lemons. So kinda tying it altogether, and then again station 3 is bringing it back to this question [frosting problem] and on the back are the reflection questions. So, based on stations 1 and 2 whatever order you did them in, now go back to the "do now" question...or see if you changed your mind? And why? Again the big thing is always why and what the reason is behind everything.
Although teachers expressed concerns about student reasoning, no specific explanations or justifications were discussed.

Lesson implementation
The lesson began by revisiting the L5 problem. The lesson's station format was well-structured. Sam kept all student groups engaged throughout the lesson with warm banter and infectious enthusiasm for the students and their efforts (reminding them that he was not going to simply tell them the answers).
For the frosting problem ("Which of the graphs represent a set of recipes that all make the same color frosting?"), Sam re-voiced the task as, "What are some features on the graphs that you might look at to determine which would be a matching pair to the situation?" This redirected the focus from the meaning of ratio equivalence(maintaining the same color of the icing) to features of a ratio graph. A student said, "It goes through the origin and it's straight." Station 1. The focus in the single audible episode of group work on station 1 was on the second task with a ratio table of boys:girls. Two students put the girls on the x-axis; the others put girls on the y-axis. Sam discussed whether it was okay to put girls on the x-axis even though they were listed in the bottom row of the We interpret this exchange to mean that the student believes that a line not being straight would interfere with making the correct prediction in the relative number of drops to keep the ratio equivalent. Sam did not ask student 2 to explain, missing an opportunity to promote learner-centeredness. This often occurs when teachers, excited by the possibilities in a student's utterance, unintentionally overwrite it with their more expert understanding, instead of fostering the emerging expression of the ideas.
Sam moved on: OK so, the recipe is going to be different, is going to be consistent going up of [sic] 9 cups of frosting, 27 drops. If you were to think about a simpler recipe, what is a simpler recipe that you could have made to make the same color frosting?...If I want to decrease the size of the frosting, 9 cups of frosting and 27 drops. Talk it out with your group, is there a smaller recipe?
The students generated two choices: 9:3 and 3:1. They labelled these ratios as "equal" and "proportional", after which Sam said: Proportional. So, if I am looking at these curved lines, am I increasing by 3 cups 1 drop, 3 cups 1 drop? Is that happening, consistent relationship here when it is a curved line? [He draws in the air stair steps of 3:1; accidentally reversing the units of measurement] This is the first time that Sam appealed to the underlying rate of change in the quantities (increasing by 1 cup for 3 drops of food coloring) as a means for preserving ratio equivalence. To describe the straightness, he introduced the phrases "to be consistent going up" and "a consistent relationship." He used a visual intuitive gesture representing slope or steepness which led to ruling out option D as "not consistent." In addressing the graphs that do not intersect the origin, a student rejected E (y = 27) because "the graph comes straight out of 27". Sam's first response interfered with this line of reasoning, simply asserting that it does not pass through the origin, but then he regrouped to follow the students' reasoning asking "What does [the horizontal line] mean in terms of the story?" One student responded, "It means that you're going to get just more drops of food coloring, and the same amount of…frosting. Again, a student spontaneously uses a rate of change description, more food coloring (change in one variable) with the same amount of frosting (no change in the other). The student set up to conclude that the frosting will be changing color. Sam instead chose to consider only the y-intercept (0 cups of frosting, 27 drops of food coloring) and encouraged students to see, in an amused way, the cake would be dripping red with food coloring. We interpret this exchange to further emphasize the spontaneity of looking at graphs in terms of growth or change and to demonstrate how important this concept is to understanding L5 as producing [straight] rays. Sam next noted that options A and F remain, two graphs with their axes reversed. He launched the discussion by saying they looked different to him and "[graph A] comes off steeper than [graph F], the first mention of "steepness" during the class. Asked if they are both correct (and why), many students responded that they were opposites and one student said, "Because they're just the same thing, they're just switched around, they're both going through the origin, and they're still a straight line." Sam: I love it, so do they still maintain this ratio that we brought in before, as to why the curved graphs were wrong? Is it a consistent ratio going upwards that creates that straight line for both of them? Is that true?
We would argue that the whole exchange supports the movement in the LT from level 4 to 5 and the underlying meaning of straightness in level 5. The students spontaneously appeal to descriptions consistent with covariation as a means to analyze why a curve disrupts straightness, but Sam focuses on the equivalent ratios, identifies the base and unit ratio, and shows how they are used to create a consistent curve.

Debrief
The debrief focused on participation goals instead of content-based learning goals. Teachers generally regarded the lesson as highly successful due to high levels of student participation. The 6th grade teachers, especially, poignantly expressed excitement about seeing quiet students from the previous year actively participating.
To obtain a crude estimate of the content dimension of the debrief, all sentences pertaining to mathematics (including observations of students' mathematical actions) in the debrief transcript were marked. The total word count of the marked sentences comprised 32% of the transcript's total word count; thus about one third of the debrief was explicitly devoted to mathematical content of the lesson.
The majority of content-based discussions involved two topics: (1) how students came to believe that two graphs with reversed axes represented the same ratio relationship; and (2) how often students chose to describe the ratio tables in terms of a constant of proportionality. In discussing axes reversal, teachers focused on students making connections between the quantities and context, learning to take more care in associating quantities with coordinate names, and paying closer attention to scale on the axes. When discussing the use of the constant of proportionality, a 7th grade teacher reported how one student related it to "unit rate".
Teachers made observations related to the LT only twice. One focused on how students completed a blank ratio table for Station 1, problem 4. One group started with the given lemonade ratio (10:8) and created recipes to make more lemonade (L1); the other opted to make less (L2) and wanted to sequence the pairs in the table from small to large. One student insisted on leaving the left-most column in the table for "the smallest recipe". A 6th grade teacher recognized this as a base ratio, a subsequent construct in the MM map.
The supervisor initiated a second discussion near the end of the debrief, directing the teachers to share their thoughts about the frosting problem: Supv: You [Sam] asked them a lot of really hard questions when you pulled it all together; like, okay, it's a straight line, so what, right? Yeah. So, so why is it that, that you're saying that this one's a curve, and it needs to be a straight line, well why? Gr. 6 Tchr: One of the boys at the middle table, I heard him say this a lot, he was saying "for every cup I add," and we use that language a lot,... Gr. 7 Tchr 1: And we use it a lot. Gr. 7 Tchr 2 : Interpret it. What does the rate mean? What does ratio mean? Supv: The interesting thing about that is they were all, they all had that idea, right? [but]they couldn't articulate it. I think the one kid said, well there's more...yeah but it's more, but it's not the same more like it's like one's more, but the other one isn't the same more like they were trying to say if one increased the other one that had to increase by a proportion. They didn't have the terminology...to put what their ideas. And that was really hard. And I think it's hard for us. I think it's hard for anybody. This is...not easy. Tchr (offscreen):...they come into the 8th grade, and the idea of slope is so foreign to them and say they have never heard this, but they have.
The discussion indicates that these 7th grade teachers recognized when a student had used the "for every cup, I add" language that is associated with unit ratios, and then saw it as a form of rate. The supervisor emphasized the struggle associated with verbalizing the shift from describing two quantities as a mixture to describing the change in quantities as a rate. Then a teacher recognized this as a transition to slope. Both examples represent teacher observations of students' emergent behavior that represents potential transitions to new levels or constructs prior to formal introduction of the ideas. The unit ratio and unit rate language reported could have also provided opportunities to connect patterns in the tables (L4) to straightness in the ratio ray (L5).

Interview
Sam's goals focused on level 5: (1) to recognize ratio equivalence in tables and graphs and thus address students' overly procedural understanding of L5, and (2) to build flexibility in reversing axes. We present Sam's expression of the first goal: ...the purpose of what I did I think was going beyond just the procedural and them being kind of robots of this, this straight line origin. And actually having them understand [what] these graphs and ratios represented. That was truly the goal of my lesson. So if I was to say my one big overarching goal, was that.
Since Sam's goal for students to "understand [what] these graphs and ratios represented" seemed to connect with four tasks at station 1, we followed up by asking about his expectations of students' responses to the question "how do you know?" if the tables represent ratio equivalence. He responded: A lot of them [the students] would think about it as doing, setting up the ratio as a the 4 to 6[sic], 12 to 8, 18 to 12, or 24 to 16 or some of them did put 4 to 6, 8 to 12, 12 to 18 and 16 to 24 and tried to reduce the fraction to all [as] two thirds. Others I know, cross multiplied every single one of the members of the table, um, others divided...6 divided by 4 or 4 divided by 6. There were a lot of different strategies in terms of the multiplicative or, uh, divisional relationships between them. ...throughout the unit we really emphasize this idea of, cause it's very big in Big Ideas [textbook] as well as I know we've seen it through state testing. That's a constant of proportionality. Um, and the concept is y over x.
We also asked Sam a follow-up question to see if he had considered using approaches from the PD that leveraged covariation patterns in the table (L4) (Fig. 6). We asked this in the context of how he and the students had explored why curved lines in the frosting problem did not represent ratio equivalence. Asked specifically whether having students simply plot points from ratio tables represented a sufficient understanding of L5, he responded: It's a tough question, because do I think they understand that it is a proportional relationship? I do. I think it is an understanding of it. Do I think it's necessarily as deep of an understanding as it could be in terms of why? I don't know about that. I think that it is definitely an understanding and I think it's more than going into a, well I have no idea what this represents, so it definitely has some levels of an understanding. I'd almost say there is a higher level than the level 5 in terms of saying like, okay, now interpret what the graph means and what do the points on the graph, why is it in a ratio, and something like that. ...I think there could definitely be more of an extension to that. Do I think that I could get more from them? Yes. Do I think it would have been accomplished in that lesson? No. Do I think it's something to revisit afterwards? Absolutely. As an extension, I will say I get a lot of this out of them. It's, it's tough. I mean these are my enriched [regular] math students. They definitely thought really tough about this stuff, but I saw a lot, I got a lot out of them that I was really, really excited about.
We suggest that Sam recognized the value of using informal references to rate of change, but he viewed the use of the stairsteps informally as a "next step" towards a formal introduction of slope. Despite some evidence of students' spontaneous observations about patterns of change, both in class and as reported by other teachers in the debrief, Sam doubted whether more could have been accomplished by regular math students in the lesson study.
Sam's second goal was to develop students' flexibility in how they could choose to display variables on graph axes: But when it's dealing with words in a ratio, whether it's boys, girls, lemonade to our lemons to sugar water, they can be interchangeable. And that was honestly my biggest goal of the lesson... ...I'll be honest in creating this, I did use the fifth level question kind of as my guiding purpose. I think I noticed a lot of the kids who got it wrong picked only one of the correct answers...So that really stood out to me as well. They're not understanding that it can be swapped and have that same relationship...That was really why I did it.
Sam acknowledged that his interest in tasks on axis reversal was due to student errors identifying the two correct responses with reversed axes in the frosting problem.

Discussion
To answer the research question, we applied our LT-DDDM framework to analyze the results, seeking to consider the coherence of the lesson study as a cycle of inquiry.

Component 1 (Scanning the data)
Although the teachers examined district data during the PD (Fig. 10), no direct reexamination of it was referenced in planning, beyond the acknowledgement of weak performance on L5. Starting at L3, approximately 40% of students were unsuccessful, increasing to 50% for L4 and 60% for L5. Teachers clearly focused the lesson design on L5 with the exception of problem 4, station 1. Teachers had only a cursory discussion of L1-L3 data. They mentioned a concern that weaker students were susceptible to the additive misconception, but did not address this in lesson study materials.

Component 2 (Interpreting the data)
In analyzing the results, we surmised that the teachers' had made two conjectures on how to improve their instruction based on their MM data.The conjectures were drawn from Sam's two goals for lesson study, combined with other expressions of goals of relevant tasks, a task analysis, and Sam's responses to follow-up questions.
The first conjecture was: "based on the data for the ratio equivalence LT, showing weakness in level 4 (tables) and L5 (graphs), students will improve learning if they develop more proficiency in recognizing representations of ratio equivalence in tables and graphs." We wrote this conjecture intentionally broad to reflect teachers' imprecise articulation of task goals such as "how you know from a table…" and "how you know from a graph…" When teachers discussed the LT in the planning meeting, except when reading verbatim, their references to data for the lower levels were non-specific and never involved examination of items beyond those used in the PD. Further, they referred to these levels as simply being about doubling and halving, a shorthand that may have led to an underestimation of the distinctions in LT levels. More precise conjectures arise from clearly articulated goals and precise descriptions of anticipated student strategies and explanations.
We used this conjecture to examine the four station 1 tasks. Their task order represents the pedagogical technique of gradually removing scaffolding, in the hope that learners will reproduce the scaffolded approach without the affordances of explicit scaffolding. This contrasts markedly with the constructivist approach modeled in the LT. The lesson study tasks also did not require students to explain how the two tests of proportionality were related, so the tasks were unlikely to address Sam's concern over rote and procedural student understanding of how to look for proportionality in graphs.
Component 2 also involves examining the data and the LT content in light of past curricular and instructional experiences. Sam's discussion of how students were expected to determine ratio equivalence shed light on this question. He expressed an expectation that, presented with horizontal tables (as typical in their textbook), students would a) test for equivalent fractions among x n / y n , b) seek a common "constant of proportionality" among y n / x n , or c) use cross multiplication (i.e. x n * y n+1 = x n+1 * y n ). These tests rely on examining the correspondence relationship between quantities (instead of covariation). Using this relationship to establish proportionality in the table, students would plot points to check if the graph was straight and intersected the origin. We do not know if teachers consciously made the shift to 7th grade content, and hence to a more sophisticated LT.
This shift was unwarranted from the standpoints of both the data and underlying LT framework. The 7th grade class data on ratio equivalence showed that a third of the class had difficulties from L2 and higher, suggesting that a correspondence approach did not remediate their difficulties in this LT (Fig. 11). So, it seemed highly unlikely that reinforcing the 7th grade approach (correspondence) would resolve student difficulties.
But perhaps the teachers had in mind another way to justify testing for ratio equivalence by simply writing y over x. Sam offered a student's explanation, i.e. that they were just checking for equivalent fractions. Researchers have shown it is mathematically unwise to use equivalent-fraction reasoning as the model for the correspondence relationship in ratio. The correspondence relationship, best described as y = kx (Vergnaud, 1988) is a referent-transforming relationship (i.e. changing from cup of sugar water to number of lemons) (Schwartz, 1988). Fractional equivalence is not; it involves partitioning the part and the whole.
This drives back to the second reason why a choice to use approaches from a more advanced LT is ultimately illadvised. More advanced tools come from more advanced generalizations which are built on simpler and more accessible ideas. Building ratio from the recipe context and actions of making more and less of it lead to constructing a table structure in which covariation follows from the original multiplicative actions for creating equivalence. Visualizing covariation in graphing helps explain the common characteristics of graphs of ratio equivalence. The shift to apply advanced tools may have been appropriate for students who had already mastered the earlier LT. However, it probably did not advance the reasoning of the struggling students and may have encouraged them to solve those problems procedurally or enact a misconceived appeal to fractions.
The teachers' second conjecture can be stated as: based on the data from the ratio equivalence LT showing difficulties in the frosting problem at level 5 (graphs), students will improve learning if they develop flexibility in recognizing that the graphs of ratio relationships with the axes reversed would be equivalent." This conjecture is more narrow and precise than the first, and, according to Sam, very closely connected with his interpretation of the item analysis on the frosting problem. It was central to the lesson design: targeting level 5, it led to using the frosting problem to open and close the lesson and to the development of station 2. The placement of variables on the axes became a focus of the discussion on items 3 and 4 of station 1. Thus about 75% of the lesson materials were directed towards it. We offer two observations concerning this conjecture: (1) the data analysis of the item options was incomplete, and (2) the conjecture relied on one item.
The frosting problem had six answer options: two curved graphs, two straight graphs that did not pass through the origin (one with a positive slope and one with zero slope), and two straight lines through the origin but with reversed axes. 25% of the district's students (n = 235) and 44% of Sam's lesson study students (n = 23) picked option A but not F, or F but not A. So, there was some support for Sam's interview observation that many students failed to recognize the equivalence of option A and F. However, a complete review of the data would have revealed that only 36% of the district's students and only 3 of Sam's students received full credit for the item. Other answer options represented credible distractors. 56% of Sam's students selected one or both of the curved graphs and 56% selected one or both of the graphs with non-zero y-intercepts. Unfortunately, these aspects concerning student reasoning of level 5 were ignored in the new task design.
Sam was more immediately concerned with the two correct options (graphs with reversed axes) in the frosting problem. We agree with Sam about its importance but would suggest that flexibility in axis reversal is a graph literacy topic. It is important to address, perhaps even as a precursor to the ratio equivalence LT, but it does not serve the purpose of trying to assist students in learning about graphing within the ratio equivalence LT. In interpreting data and developing conjectures, one needs to identify the most significant and relevant targets for intervention based on the data and LTs.
Our final concern resides in the single-item focus. In the framework, we emphasize the importance of situating a problem within the hierarchical LT structure, because we have frequently seen teachers' discussion of an item inordinately focused on unique characteristics of the item, instead of discussing it as an instance of a level or an LT. This is a habit often formed when teachers are accustomed to tests made by domain sampling.
Because conjecture 2 proved to be unrelated to the LT, we only investigated components 3 and 4 related to conjecture 1.

Component 3 (Testing conjectures and modifying instruction)
Sam implemented his modifications to instruction by creating stations where he told us he had grouped students into weaker and strong performing students and assigned the stronger students to begin with station 2. Further, he said he placed one particularly articulate student in each group to stimulate discourse. It was clear that students were more engaged when tasks required active student contributions.
In this component, one examines instructional modifications as a test of the conjecture by viewing instruction through the lens of the conjecture. Dialectly, one also views the conjecture through the instructional events.
For the first conjecture, relevant instruction took place in station 1 and during the review of the frosting problem. Unfortunately, the videos did not capture work related to problems 1 and 2. Sam was heard discussing the placement of variables on the axis on problem 3, but not ratio equivalence. The only evidence of student activity concerning conjecture 1 was the observation by the teacher during the debrief that she observed a group reasoning with making more and making less on the lemonade problem.
Students reworked the frosting problem to begin the class. As reported, Sam revoiced the problem shifting from "finding the recipes that make the same color frosting" to "what are some features on the graphs that might look for the match". In doing so, he likely reduced the cognitive demand of the task, shifting it away from interpreting how to represent in a graph to looking for a set of features. Evidence of this came as one student was heard dutifully reciting the memorized features associated with graphing.
However, in the final rushed minutes of the class during the discussion of frosting problem, there was evidence of students actively engaging with what it means to graph ratio equivalence in response to Sam's able questioning. When a student says non-straight graphs cannot match the situation, Sam asks what it means to be "non-straight"? Later, when a student describes the graph as "coming straight out of 27", Sam asks the student to "tell the story of that graph". Both questions, following student observations, support students in pursuing their ideas, an essential quality of learner-centered instruction.
We note, however, while conjecture 1 draws attention to how to see or test for ratio equivalence in graphs, it is silent how to support it. While Sam expresses the need to address how procedural their understanding is, no suggestions were proposed. The breadth of the conjecture meant there was a lack of specificity about how to promote an understanding of equivalence in graphs. However, student comments on the MM assessment problem (frosting problem) catalyzed the necessary discussion. In answer to what it is to be nonstraight, two student comments, "one's gonna have more or something, the other is gonna have.." and "It could interfere with other numbers that make the recipe different." show that students are inclined to visually analyze straightness in terms of the connections between points. The description of a graph as "coming straight out of 27" also expresses this visual attention to straightness. We suggest that a careful analysis of these students' contributions can provide insight into how to support students' deeper understanding of why ratio equivalence produces straight lines. In fact, in responding to these student descriptions, Sam actually does describe equivalent ratios for (27,9), until he reaches (3, 1). He then describes this equivalent ratio as a means to ask whether the curved line can be described as "increasing by 3 cups, 1 drop, 3 cups, 1 drop' gesturing stairsteps. In this exchange, we see Sam spontaneously also appeal to the "consistency" connected with covariation in ratio equivalence which can subsequently evolve into rate of change and slope.
We highlight two key insights: (1) By looking at the conjectures from the perspective of learner-centeredness, one recognizes that had the teachers been more specific in making their conjecture (by indicating expectations for establishing and relating tests for ratio equivalence in both representations), then they could have addressed the procedural problem.
(2) These episodes demonstrate how understanding the structure and meaning of LT levels can help one to recognize and build on emerging student ideas.

Component 4 (Reassess and Review)
The fourth component involves reassessment followed by review of one's efforts to use and interpret data in pursuit of improved learning. After lesson study students were reassessed and we compared the results of the pre-and posttest (Fig. 11). The average pretest score was 23% (s.d. 16%). To make a fair comparison, we eliminated the scores of students who did not take both tests. For the remaining students(n = 16), the average pretest score was 26% (s.d. 18%) and the average posttest score was 56% (s.d. 21%), a gain of 30%. We examined the heatmaps and observed little to no improvement on L2, L3, and L5, and improvement on items at L1 and L4. L5 items were the only items that could have been directly affected by the instructional tasks that check for axis reversal. Students' responses in the posttest item responses still showed evidence that students were still reversing the values in the coordinate pairs. Whilst the test score gain appears substantial, the low posttest score on a 6th grade test suggests students have not yet gained full proficiency in ratio equivalence and reached the top level, as hoped.
We engaged in our own reflection of the pathway connected with this case study, looking for coherence, alignment, and evolution across the components in the LT-DDDM framework and summarize our reflections as: 1. Teacher data scans did not attend sufficiently to weak performance on the lower levels of the LT and the persistent additive misconception. 2. Of the two conjectures only the first, about recognizing equivalence in tables and graphs, was properly related to the LT and its data. The second conjecture focused excessively on only on two of six options within a single item. In choosing that narrow focus, critical elements of the LT were not addressed. 3. The first conjecture needed revisions if it was going to successfully connect the way of examining tables for equivalence with that for graphs. Covariation could have been leveraged to address students' procedural understanding of straightness in ratio graphs. Tasks could have been designed to support an intuitive introduction to rate of growth and slope and hence, offer a conceptual grounding for the constant of proportionality and y = kx. 4. The sequencing of the station 1 tasks should have been reversed to support students to actively construct the tables and graphs. Had this been done, the introduction to correspondence would have been delayed until basic ratio equivalence was understood along with a mastery of successive LTs: base ratio, unit ratio, and building up and down with ratios.

Conclusions
The study was designed to shed light on the instructional change approach that underlies the use of a diagnostic formative assessment system based on LTs. The process of documentational genesis observed confirms the insufficiency of simply evaluating MM's use in terms of implementation fidelity. Rather, what is required is a more nuanced and evolutionary view of the tool's use as a scheme for utilization (Rabardel, 1995) such as is envisioned as documentational genesis (Gueudet & Trouche, 2009). Teachers inspired us to think carefully about what is involved in using MM data to become more learner-centered. Our analysis demonstrated how easy it is to treat data too casually, acting as if they compel a course of action sans rigorous interpretation. The teachers did use the student outcome data from the workshop, combined with their goal to engender more student participation, to propose a set of tasks for a lesson study. In doing so, we showed how they transformed the data in two ways which were fundamentally inconsistent relative to the LT. On tasks evaluating ratio equivalence, they reversed the sequence implied by the LT by giving completed tables and graphs before asking students to build ratio tables. This approach limited students' constructive activity. On L5 where students showed evidence of considerable weakness in working with graphs of ratios, they designed tasks to focus on developing students' flexibility in displaying data rather than on understanding the features of ratio graphs. These both represent examples of teachers using data but not using them to achieve the goals of the LT. Further, we also documented the tendency of the teachers to discuss pedagogy in the absence of content, to remain vague about the goals of the tasks and the expected behavior of students.
In conducting our documentation of this process of instrumentalization, we recognized that our own assumptions about the demands and requirements of successful data use needed further clarification and explicit articulation. We formalized our process of LT-based data use into a framework that allowed us to analyze and illustrate what went awry in this case. Our framework emphasizes the need for: (1) a careful process of analyzing data patterns both at and between levels, (2) forming those into conjectures that relate to current curricular materials use and to a teachers' knowledge of her students and their needs, (3) testing those conjectures in subsequent instructional changes, and (4) evaluating the success of those changes. Thus, a contribution of this article is its articulation of a data-use framework for using data from a diagnostic assessment of students' progress along learning trajectories that can be applied by others working on LTbased classroom assessment. While this study has limitations in that it represents only a single case study of one instantiation of the system's use within a school district, subsequent studies will allow further elaboration of the framework.
By applying the framework to the case, we articulated how our LTs are related to learner-centered instruction. For teachers to leverage a LT-based approach, they must be committed to helping students build from their concrete experiences in a constructivist manner. The teachers' reversal of ratio equivalence tasks undermined this assumption, hence this characteristic of the LTs needs more attention. Secondly, the tendency of the teachers to view L4 and L5 as two separate behaviors without seeing how the covariation patterns in the tables lead to the features of ratio graphs (straight lines through the origin) indicates a lack of understanding of the way the sequence depends on ideas of genetic epistemology and guided reinvention. Thus, we conclude that teachers must learn to trust the LTs and their underlying learning theory in constructivism.