What Have We Learned through the Gates-Funded Teacher Evaluation Reforms?

Nine years, $575 million dollars, and 500-plus pages later, what have we become experienced in the Gates Foundation’s ambitious efforts to better teacher effectiveness through evaluation and human capital reforms? The headlines around the RAND Corporation’s recently released final report have focused on deficiency of any consistent effects on student outcomes, nevertheless the real story here is the many insights about implementation-what actually happened about the ground-based on rich qualitative and survey data.? Here are some of my key takeaways with the report.

The study evaluated the Gates’ Intensive Partnership initiative with three school districts and 4 charter management organizations, which lasted from 2009 to 2015 and provided $575 million in complete funding ($800 – $3,500 per pupil). In return, participating districts/CMOs focused on implementing major reforms to the teacher recruitment, screening, evaluation, and compensation systems.

In plenty of different ways this initiative needs to be considered an evidence of concept. The participating districts/CMOs were specifically selected with regard to their strong resolve forpersistance to the reforms, they usually had unprecedented financial support. Their efforts give a rare window into whether evaluation and human capital reforms work under very favorable circumstances-a truer test in the reforms themselves.

Despite the strong initial buy-in and generous funding, capacity constraints along with the tension between using evaluation both for formative and summative purposes been shown to be major implementation challenges. The evaluations were an incredible burden on principals as well as new system generated few teachers rated below Proficient. The consistent upward change in ratings after some time was surprising for me. I originally thought one other happens in the event the first new evaluation ratings came out not long ago. I believed there would be an implementation learning curve, but I underappreciated the interpersonal aspects of the evaluation process. Principals consistently reported their preferences for helping teachers improve versus dismissing them. This included recognizing and rewarding effort/growth in evaluations. However, rewarding marginal growth with the additional point on a four- or five-category scale quickly generated inflated ratings. It’s unsurprising then, that teachers were rarely rated below Experienced in the new system.

The report also learned that districts/CMOs struggled to implement the evaluations as a possible engine for professional development. Tries to differentiate professional development dependant on individual performance by giving teachers use of on-line clearinghouses of PD resources were considered as ineffective by teachers. The observation and feedback process was limited because of capacity and funding constraints. Frequent individualized feedback ‘s time intensive, and time is really a major driver of personnel costs, which constrained the utilization of feedback as a development tool. Yes, amazingly, it seems that system-wide improvement via observation and feedback requires far more investment (or otherwise better utilization of existing resources.)

Twenty years from now, one positive legacy of the reform initiatives are going to be the widespread using rigorous classroom observation rubrics. I thought this was mostly of the elements that is viewed favorably by both teachers and school leaders well within the reform. These next-generation rubrics offer a clear benchmark and common language as to what high-quality instruction resembles.

Another key goal would have been to attract quality teachers and bolster retention through bonuses and career ladder opportunities connected with evaluations. Even though the logic is actually apparent, the limited implementation of the lever had been a real missed opportunity. Teachers saw the job ladder opportunities more akin to “add-ons” in place of distinguished job promotions. In addition, they repeatedly expressed that this modest bonuses-on the transaction of some thousand dollars-weren’t enough to motivate a general change in practice.

Perhaps the greatest lost opportunity on the initiative was unable to provide sufficient incentives to teachers considering the available funding. The underpowered efforts never change teacher practices or even to attract high quality novice teachers. One school leader is quoted as writing “Our teaching staff will be as good as might know about should select. We must strengthen our pool of eligible candidates . . . ” Supplemental bonuses and secondary add-on roles are unlikely to become very visible or efficient way to recruit new talent into teaching. A full redesign of teacher compensation with higher top-end base salaries may have provided much greater insights with regards to the potential of these reforms for attracting and retaining quality talent. Concerns concerning the sustainability of such a reform likely played a role here, but you can still find many different ways to examine compensation reforms that happen to be cost-neutral.

It is?not surprising, then, that RAND found no evidence the districts/CMOs could attract excellent quality novice teachers. Evaluation reforms introduce new risks in to the teaching profession and, now and again, can make teachers appears like they’ve got more limited professional autonomy. Meaningful career growth opportunities and potential earnings might have offset these potential negative consequences.

The report also highlighted a rarely discussed headwind during the evaluation reform era: the battle caused from the substantial turnover in district/CMO leadership. The achievements evaluation and human capital reforms in D.C. is not understood not in the context within the consistent leadership they’ve got benefitted from throughout the last decade.

On a positive note, two key takeaways regarding placement and school turnover left the report. The initiative led to a meaningful lowering of both forced teacher placements and seniority transfers. Therefore tenured teachers cannot secure a posture were less likely to remain foisted onto schools understanding that less-experienced teachers were “bumped” using their company positions by more senior teachers less often.

In terms of the impact analyses, the moderate-to-large positive results reform advocates had envisioned can be eliminated. It is not easy to state extra with any precision given several important limitations in which the authors recognize. Three issues stand-out as particularly concerning:

? A good sample height and width of only seven districts/CMOs
? This brief pre-treatment period which limits the chance to project counterfactual trends while in the participating districts/CMOs
? The confounding effect of high-stakes teacher evaluation reforms occurring across the nation concurrently using the initiative

It is probable that this outcomes of the Gates’ Intensive Partnership initiative will likely be seen simply when the final nail in the coffin within the teacher evaluation reform era, but they offer a great deal more nuanced insights. Despite underwhelming impact findings and real implementation concerns, the initiative resulted in a very rich list of more knowledge about the organizational and cultural barriers to enacting major human capital reforms in public places schools. Instead, I’m hoping these studies informs how districts and CMOs invest their future resources and build new systems to draw, develop, and retain effective teachers nowadays in this ESSA era.

Alex Bolves and Alvin Christian provided excellent?assistance?in some recoverable format this text. Matthew Kraft is undoubtedly an assistant professor of education and economics at Brown University.? Ask for about his research at www.matthewakraft.com and follow him on Twitter @MatthewAKraft.