Dexterity, Deliberateness, And Disposition: An Investigation of Instructional Strength for Early Literacy

This mixed-methods study examines the relationship between an emergent conceptualization of teachers’ instructional strength and their students’ progress in early literacy. The conceptualization includes three components: a teacher’s deliberateness, their instructional dexterity, and a set of teacher dispositions that catalyse and maximize these attributes. Based on the results of qualitative inquiry, the authors developed a measure of individual teachers’ instructional strength according to this conceptualization. Regression analysis reveals that all three components are significant predictors of students’ growth in early literacy. The study includes 318 teachers and 1,181 students from 227 schools across the country.


Main Text Introduction
The quality of instruction provided by teachers is a critical and variable determinant of student learning (Aaronson, Barrow & Sander, 2007;Araujo et al., 2016;Darling-Hammond, 2000;Hanushek, 2011;Rockoff, 2004), and the "proximal cause of student progress" (Raudenbush, 2008, p. 207). The work presented here builds on existing educator-effectiveness research by conceptualizing and assessing instructional strength, defined as the extent to which a teacher instructs for maximum learning in every lesson (Authors, 2016). We then examine the association between this conceptualization of instructional strength and student achievement in literacy.
Our findings are the product of multiple years of mixed-methods research. Some of this research was conducted as part of a large-scale evaluation of Reading Recovery, which included a rigorous, large-scale randomized trial and a deep focus on implementation. The model of instructional strength we examine here was developed, in part, in the course of that research. It posits that strong instruction can by recognized by the extent to which it exhibits two characteristics: deliberateness, which we define as an encompassing commitment to thoughtful practice; and dexterity, which we define as the flexible application of deep skill. The model further suggests that deliberateness and dexterity can best flourish when teachers also demonstrate key dispositions that incline them to make the most of their training and instructional resources. This paper expands upon that earlier work by examining, for the first time, the relationship between our conceptualization of instructional strength and student learning. While findings have immediate relevance for Reading Recovery scholars and practitioners, who number in the tens of thousands worldwide, this framework may also offer a starting point for a widely applicable perspective on teaching that has implications for teacher preparation.

Prior research on teacher effectiveness
Our conceptualization of instructional strength builds on several growing literatures. First and foremost, it builds upon research on educator effectiveness. This literature is broad, spanning efforts to identify the attributes of strong teachers, to explore the nature and development of effective teaching practice, and to quantify the impacts of high-quality instruction on student learning. One major strand of the teacher-effectiveness research seeks to identify relationships between student achievement and proxy indicators of teacher quality--for instance, teacher characteristics like years of experience, training, and credentials (Cowan & Goldhaber, 2016;Darling-Hammond, 2000;Goldhaber & Anthony, work ethics, feelings, morals, behaviors, skills, intentions, competence, or values" (2014, p. 25). Our own definition imagines teacher dispositions as inclinations-that is, as ways of approaching students and teaching that result from a combination of innate and learned perspectives and characteristics. We do not attempt, here, to discern the precise origins of these inclinations. Rather, we are interested in better understanding their manifestation in strong instruction and, ultimately, in the relationship between teaching and learning. In exploring all of these aspects of instructional strength, we address the following research questions: What are the characteristics of the strongest Reading Recovery instruction? What dispositions incline teachers to deliver strong instruction? And, does a teacher's instructional strength predict her students' growth in reading?

Context of the Research
Reading Recovery is a decades-old literacy intervention that is widely used in the US, Australia, New Zealand, the UK, and elsewhere. The intervention is delivered to struggling first-grade students by teachers specially trained to implement individualized teaching plans for each student (Clay, 2005), and unfolds over the course of several months of daily, individual, 30-minute lessons. Reading Recovery's basis is in constructivist learning theory (Bruner, 1966(Bruner, , 1983Dewey, 1916;Piaget, 1967), which views learning as an active process in which students are responsible for constructing meaning based on their own beliefs and experiences.
Constructivist theorists-Reading Recovery's creator Marie Clay among them (Clay, 1991(Clay, , 1998-regard the teacher's role in the learning process as primarily concerned with providing the student with opportunities to learn. A skillful teacher, from this perspective, is adept at anticipating, identifying, and arranging learning opportunities (Clay, 2005).
Reading Recovery lessons follow a prescribed pattern. Each includes a series of activities that progress in a particular order. Within this structure, the teacher exercises her discretion-informed by Reading Recovery theory and training-in selecting the instructional foci, texts, and teaching moves that best suit each student. Students' progress and lesson activities are carefully tracked via running records and other lesson documentation (Authors, 2018). Because of this program model-which embeds idiosyncratic teaching within a consistent structure or frame-Reading Recovery offers an ideal setting for close study of variations in instructional technique.
An important role in this research is that of the Reading Recovery teacher coach-or "teacher leader" in Reading Recovery parlance. Teacher leaders teach the year-long university-based courses Reading Recovery teachers are required to complete as part of their training, and support the teachers through intensive, ongoing, on-site coaching that includes regular observation of instruction. Nearly all teacher leaders are experienced Reading Recovery teachers themselves, with deep knowledge of the program's lesson structure and theory. They are the primary sources of the data we used to conceptualize, operationalize, and measure instructional strength.
This research was conducted, in part, within the i3 evaluation of Reading Recovery. Funded by a grant from the U.S. Department of Education's Office of Innovation and Improvement's (i3) Scaling Up What Works program, the evaluation was conducted between 2011 and 2014. The current study grew out of the observation that Reading Recovery lessons vary in important ways that have meaningful consequences for student learning. This observation echoes with a major finding of the randomized trial component of the evaluation: While Reading Recovery students outperformed control-group students in reading overall, some students' growth was particularly remarkable (Authors, 2018).

Materials and Methods
Data for this study come from interviews, observations, and focus groups with Reading Recovery practitioners, as well as student-level data achievement data, were collected in the course of the i3-funded evaluation. Following the conclusion of the impact evaluation, we developed a measure of instructional strength and administered as an online survey to teacher leaders who worked as a mentor with teachers in the study. Finally, we conducted regression analysis to examine the extent to which these individual teachers' instructional strength scores, as calculated from their mentors' survey responses, predicted the reading achievement of their students.

Sample
Participants in the research included Reading Recovery teachers, teacher leaders, and university-based regional supervisors, as well as first-grade students who received Reading Recovery lessons during the 2013-2014 school year (See Table 1). All teacher leaders and regional supervisors involved in the i3 scale-up were contacted for data collection activities, and all who responded were included in interviews and focus groups. Teachers in the study were randomly selected for participation in interviews.
* Data collected during the i3 evaluation.

** Data collected following the conclusion of the i3 evaluation
In the summer of 2014, after the conclusion of the impact evaluation, we surveyed 120 teacher leaders who had supervised 318 teachers during the 2013-2014 school year. These teachers and students were from 227 schools across the country. We then used the resulting data on these teachers' instructional strength, along achievement data from 1,181 of their students, to explore the relationship between instructional strength and student progress in reading.
The student outcome data used for this exploration were collected as part of the multi-site, delayed-treatment design of the RCT study. Once deemed eligible for Reading Recovery by virtue of their low scores on screening assessments, first-grade students in the schools in the study were matched in pairs and randomly assigned, within their pairs, to treatment and control. Students assigned to treatment received Reading Recovery in the first part of the school year; those assigned to control received it once their matched counterparts had completed the program. Both students were assessed at the beginning and end of their Reading Recovery intervention period (Authors, 2018). Student impacts used in the current study were estimated using data from students in both groups.
All participants in the research provided signed or verbal consent to the data collection. All consent procedures and data collection instruments were reviewed and approved by the University of Pennsylvania's Institutional Review Board. Student assessment data were collected with parental consent as a routine part of their participation in their schools' Reading Recovery programs, and then shared with the research team.

Interviews and observations
In the first two years of i3 study, we conducted semi-structured interviews with Reading Recovery teachers, teacher leaders, and the university faculty members who train teacher leaders and oversee Reading Recovery via its network of university-based regional hubs. At this stage, we used interview protocols designed to elicit general information about barriers and facilitators of Reading Recovery's implementation in schools. Interviews were recorded and transcribed. After reaching at least 80 percent inter-rater reliability, researchers coded the data using both general codes derived from the interview protocols and codes that emerged inductively via multiple reviews of the data (Ravitch & Carl, 2016). Through our constant-comparative analysis of these data (Miles, Huberman, & Saldaña, 2014), we identified "lesson quality" as a key implementation factor (Authors, 2016).
In the final two years of the evaluation, we conducted targeted inquiry around this theme of lesson quality. In year three, we revised the protocol for our interviews with university faculty to include questions about how Reading Recovery lessons vary from teacher to teacher and from day to day, and about what instructional practices and contexts best support student learning. Data from these interviews were prepared for analysis via the process described above; this time, however, researchers applied both general implementation codes and also sub-codes specific to instruction (e.g. "teacher prompting") within pertinent parent codes (e.g. "lessons"). The sub-codes related to instruction were developed collaboratively by the researchers through multiple reviews of the data. Through constant comparative analysis of the coded data, we identified the themes that ultimately informed our conceptualization of instructional strength.
This conceptualization was further informed by the research team's 108 field-based observations of Reading Recovery lessons. These observations were guided by a protocol we developed to record the sequence of activities in each lesson, when and how the teacher prompted students, techniques the teacher used to instruct students on specific skills, and the proportion of lesson time dedicated to each activity. Following each observation, researchers conducted a short interview with the teacher, who was asked to explain the specific instructional decisions she made during the lesson (Emerson, Fretz, & Shaw, 2011). When possible, researchers observed lessons in the presence of teacher leaders, who were also interviewed about the lesson. In combination with our interviews, these observations allowed the research team to develop deep familiarity with Reading Recovery lessons.

Focus Groups
To confirm and further refine our emerging conceptualization, in the final year of the i3 study we conducted five teacher leader focus groups. Each hour-long focus group was held in a different state and included teacher leaders affiliated with a different university-based training site. Focus groups included between six and twelve teacher leaders, each serving as instructor and coach for a group of Reading Recovery teachers involved in the evaluation. During the focus groups, we asked teacher leaders to describe, as concretely as possible, ways in which individual lessons vary and the characteristics of particularly strong lessons. Focus group transcripts were coded using deductive codes related to the hypothesized components of instructional strength, and open coded to ensure no themes were missed. We then applied subcodes for the specific manifestations and features of each dimension of instructional strength.

Survey
The results of four years of observation and qualitative inquiry informed an emerging conceptualization of instructional strength that includes two distinct but related manifestations--instructional dexterity and deliberateness--and a set of underlying teacher dispositions. Our efforts to understand this conceptualization and its significance for student learning continued beyond the end of the study with the development and administration of a survey for teacher leaders who had supported Reading Recovery teachers in the i3 scale-up during the 2013-14 school year. Teacher leaders were identified as respondents because of both their deep expertise about early literacy instruction in general, and their familiarity with the idiosyncrasies of individual teachers' instruction.
Each of the three components of instructional strength was operationalized using multiple indicators that could be administered as items in an online survey instrument. Each item asked teacher leaders to evaluate the teacher relative to an average Reading Recovery teacher using a six-point Likert scale of agreement. Items were eliminated and/or refined through a process that included a pilot administration to a group of teacher leaders and university-based trainers, followed by cognitive interviews with respondents to ensure that all items were worded clearly and interpreted consistently. Ultimately, we administered and included in the analysis eight items designed to assess teachers' deliberateness, six items about instructional dexterity, and five items about dispositions. The survey was administered in Summer, 2014, after the impact study was finished. In completing the survey, each teacher leader was asked to respond to a series of items about every teacher she supported during the 2013-2014 school year.

Regression Analysis
The final step in our research was to examine the extent to which instructional strength-as represented by the individual teacher scores yielded by the survey-predicts student achievement in reading. The outcome measure was the Observation Survey of Early Literacy Achievement (OS) (Clay, 2002(Clay, , 2005(Clay, , 2016. The OS is the standard assessment used in Reading Recovery. It is an individually administered standardized assessment comprising six sub-scales that, combined, form a unidimensional measure of early reading achievement (D'Agostino, Rodgers, & Mauch, 2018). Multiple studies have supported the validity of the OS as a measure of early reading achievement (Denton, Ciancio, Fletcher, 2006;Gómez-Bellengé, Gibson, Tang, Doyle, & Kelly, 2007;Gómez-Bellengé, Rodgers, Wang, & Schultz, 2005;NCRTI, 2012;. The OS measure was administered to all students prior to receiving Reading Recovery, and again at the end of each student's intervention period (Authors, 2018). The date of each assessment and the number of Reading Recovery lessons provided, a measure of dosage, were also recorded for each student.
To estimate the impact of teachers' instructional strength on their students' reading achievement, we regressed student-level reading achievement data on teacher instructional strength scores from the survey. We used a hierarchical linear model (HLM) in which students were nested within teachers, and teachers nested within schools. The dimensions of instructional strength reflected in our conceptual model, while related, are distinct. Each looks different in practice and, we hypothesize, requires different types of supports or interventions to improve. For this reason, we estimate separate HLM models for the three dimensions.
To account for differences in teacher leaders' perceptions and reporting, and to reflect the fact that all the Reading Recovery teachers in a given school are typically supported by a single teacher leader, we included a fourth level in the model to nest schools within teacher leaders. In the interests of parsimony, we included only two covariates in the model: OS pre-test total score and the total number of Reading Recovery lessons provided to each student. This latter covariate was included to account for the fact that some students received more lessons-and therefore a higher dose of their teachers' instruction-than others. Table 2 shows the distribution of these variables. Based on national percentile ranks from the instruments U.S. norming sample (2012), the average scores at entry and exit are associated with the 17 th percentile among students in the Fall, and the 39 th percentile at mid-year.

Results
In this section, we first summarize the results of our qualitative inquiry into instructional strength, which was conducted during the i3 evaluation (Authors, 2018). We then discuss the findings of work completely following the larger study, including our examination of the factor structure of the instructional strength construct, and our exploration of the relationship between instructional strength and student learning.

Results of the Qualitative Inquiry
As noted, our qualitative research produced a hypothesized model of instructional strength that includes two manifestations of instructional strength, deliberateness, and instructional dexterity. While deliberateness and dexterity are related, we observed qualitatively that they are manifested in different ways and at different times. We observed, further, that each is expressed through a set of concrete behaviors or practices on the part of Reading Recovery teachers, and that the strongest instruction is characterized by high levels of both deliberateness and dexterity. Our research also revealed a set of dispositions that underlie both deliberateness and instructional dexterity, and that facilitate the demonstration and development of instructional strength.
In our conceptualization, deliberateness is defined as an encompassing commitment to thoughtful practice, and we see it in teachers' consistent engagement in three types of activities: the use of careful lesson documentation for purposes of analyzing individual students' progress and planning instruction, continual reflection on their own instruction, and dogged pursuit of knowledge that includes participation in a community of practice. These activities take place primarily in the time in between Reading Recovery lessons, rather than during instruction itself. More specifically, we observe that strong Reading Recovery teachers are not just faithful documenters of their students' progress; they are expert noticers-skilled at converting subtle observations about students' strengths, weaknesses, and interests into data that inform all aspects of their instruction. Further, strong teachers take their Reading Recovery training very seriously, and attend training activities faithfully. They also access the expertise of their teacher leaders regularly, and build and routinely engage with practitioner networks of teachers.
The second manifestation of instructional strength, instructional dexterity, is defined as the flexible application of deep skill. Unlike deliberateness, instructional dexterity is primarily evident during lesson enactment. The first expression of dexterity that that we observe concerns the nature of the teacher's rapport with her students. The strongest instruction is characterized by warm but firm support, and the strongest teachers can judge when the time is right to provide encouragement, and when a student can be pushed to call upon and apply new skills independently. In doing so, these teachers help their students develop confidence and self-sufficiency. Another expression of instructional dexterity we identify is the ability to make instantaneous instructional decisions based on both prior information and in-the-moment observations. While Reading Recovery teachers are taught to make a plan ahead of each lesson, dexterous teachers can deviate from that plan in response to students' successes or their difficulties. Students' reading and writing errors present particularly rich teaching opportunities for dexterous teachers, as they know how to acknowledge the student's effort and quickly transition the mistake into new learning. Finally, we observe that dexterous teachers exhibit economy of language. They speak mostly to arrange learning opportunities or prompt students' application of skills, rather than to explain or instruct. As a result, their statements are brief, clear, and redirecting. Typical prompts by a dexterous teacher might consist of statements like "Again" or "Try a word that looks like that word." According to teacher leaders, this judicious use of language is an indication that a teacher is successfully identifying the student's strengths and development needs and arranging appropriate practice opportunities.
We also observe that underlying both deliberateness and instructional dexterity is a set of dispositions that facilitate the development of instructional strength. By and large, respondents described these attitudes and dispositions as attributes that interact with Reading Recovery training to support the behaviors we associate with deliberateness and dexterity. In our conceptualization, dispositions undergirding instructional strength include the following: Openness to change, interpersonal communication, work ethic, and belief in the ability of all students to learn.

Factor Structure
To investigate the composition and structure of the theorized dimensions, survey items were submitted to a confirmatory factor analysis (Anderberg, 1973;Harman, 1976). We found that all items had a higher squared multiple correlation with items in its theorized scale, than with the items of the other scale. Although the two scales were highly correlated (r=0.86), item membership was able to explain 82.7% of the total variance, and each scale had high internal consistency ≥.95. Item means, standard deviations, item-total correlations and conditional alphas are presented by scale in Table 3. Relationship to student achievement The study next examined associations between scales of instructional strength and student-level reading achievement data. Table 4 presents the fixed and random effects for a model that was estimated three times for each of survey scales. Random intercept coefficients were statistically significant for all four levels: Teacher leader, school, teacher, and student. Fixed effect estimates for baseline assessment and the number of lessons between measures were also statistically significant in the three models. The significant main effects for pretest and number of lessons indicates that students who started at higher reading levels, and who received greater number of one-to-one lessons, on average scored higher. The estimated effects for Instructional Strength scales were also statistically significant. We estimated a 2.5 and 2.6 increase in Total OS scores for every 1-point increase on scales of deliberateness and dexterity, respectively. These gains are equivalent to the estimated improvements after five lessons (typically 1 week of individual instruction). Fully standardized coefficients associated with these effects are 0.06 and 0.07 standard deviations. Findings offer evidence that Reading Recovery teachers' instructional strength is associated with their students' improved reading achievement. The disposition scale was also a significant predictor of student reading achievement with effect size comparable to the IS scales.

Discussion
This study addresses both the nature of instructional strength and its relationship to reading achievement. Findings suggest that the strongest Reading Recovery lessons are similar to one another in observable ways and are conducted by teachers who behave in similar ways. The students who received strong instruction, as conceptualized in this study, were observed to make the greatest gains in reading. This suggests future utility of this conceptualization in terms of understanding instruction. More specifically, our findings contribute to the teacher-effectiveness literature by highlighting a theory that not only links instruction to learning, but also offers specificity on the instructional activities that make some teachers especially effective.
All three scales in our measure showed small but significant effects on student achievement. Along with confirming our theory about the instructional practices strong teachers use, this research therefore confirms our hypothesis that particular dispositions incline some teachers to leverage their training and resources more effectively than others. This represents a contribution to prior research on the role of teachers' personal characteristics in their students' learning.
Because the goal of the i3 grant was to scale up Reading Recovery nationally by training new teachers to deliver the intervention, the sample consists largely of teachers recently trained to implement the program. Thus, one limitation of the study is that we are unable to assess whether instructional strength looks different or produces different outcomes in teachers with more experience using Reading Recovery. A second limitation is that the research that informs our conceptualization of instructional strength focused specifically on Reading Recovery teachers, who have gone through a rigorous training process in order to enact a particular type of instruction. The generalizability of our findings to teachers with different training backgrounds is unknown and warrants further exploration.
Future research may seek to address the types of training that can best support the development of instructional dexterity and deliberateness Findings suggest that training models must attend to teacher attitudes and dispositions which improve receptiveness to training and lead to instructional strength for early literacy. We hope findings from this study will advance an understanding of both the practices and attributes of strong teachers in ways that are useful for educator preparation and development.